Oct 30 20:03:26.919013 ip-10-0-129-14 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 20:03:27.272705 ip-10-0-129-14 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:03:27.272705 ip-10-0-129-14 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 20:03:27.272705 ip-10-0-129-14 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:03:27.272705 ip-10-0-129-14 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:03:27.277464 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277377    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.277464 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277461    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.277464 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277466    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.277464 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277469    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.277464 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277472    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277475    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277477    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277480    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277483    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277486    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277488    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277491    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277494    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277496    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277499    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277502    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277504    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277507    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277509    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277512    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277515    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277517    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277520    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.277677 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277522    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277524    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277531    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277533    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277537    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277539    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277542    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277544    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277547    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277549    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277552    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277555    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277558    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277561    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277563    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277567    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277570    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277572    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277575    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277578    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.278115 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277580    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277583    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277586    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277588    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277591    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277594    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277597    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277599    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277602    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277604    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277607    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277612    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277617    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277620    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277623    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277626    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277629    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277631    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277634    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.278612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277656    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277660    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277664    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277668    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277673    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277678    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277680    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277684    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277686    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277689    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277692    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277695    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277697    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277700    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277703    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277706    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277708    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277710    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277713    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277716    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.279073 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277718    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277722    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277725    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277727    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277730    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277733    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277735    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277738    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277741    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277751    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277754    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277757    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277759    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277762    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277764    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277767    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277770    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277773    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277778    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.279557 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277782    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277785    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277788    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277791    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.277793    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278568    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278574    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278577    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278580    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278583    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278585    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278588    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278592    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278596    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278598    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278602    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278604    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278607    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278610    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.280030 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278613    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278616    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278618    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278621    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278625    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278628    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278630    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278633    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278655    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278659    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278663    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278668    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278671    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278673    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278676    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278679    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278682    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278684    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278687    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.280521 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278690    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278692    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278695    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278697    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278700    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278702    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278705    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278708    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278710    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278713    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278716    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278719    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278722    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278725    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278727    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278730    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278732    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278735    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278737    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278740    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.280985 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278743    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278746    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278749    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278751    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278753    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278756    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278759    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278761    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278764    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278767    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278769    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278771    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278774    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278776    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278779    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278781    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278783    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278786    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278788    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.281498 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278791    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278793    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278795    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278798    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278801    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278804    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278806    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278809    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278812    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278815    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278819    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278824    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278827    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278830    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278833    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278836    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278838    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278841    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278843    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.281955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278846    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278848    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278851    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278853    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278856    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278858    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278861    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278863    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278866    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278868    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278871    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278873    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278876    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278878    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.278880    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279367    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279372    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279394    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279399    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.282404 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279403    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279406    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279410    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279413    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279415    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279418    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279421    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279423    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279426    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279428    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279431    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279434    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279437    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279439    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279441    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279444    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279447    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279449    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279452    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.282867 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279454    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279457    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279460    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279463    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279466    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279469    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279471    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279474    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279476    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279479    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279482    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279484    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279488    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279490    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279493    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279496    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279498    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279501    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279504    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279506    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.283318 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279509    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279511    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279514    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279516    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279518    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279521    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279523    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279526    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279528    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279531    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279533    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279535    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279538    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279540    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279543    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279545    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279548    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279550    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279553    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279556    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.283799 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279558    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279561    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279566    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279569    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279571    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279576    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279579    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279582    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279584    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279587    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279589    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279592    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279594    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279597    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279599    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279602    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279604    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279607    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279610    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279612    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.284276 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279615    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279617    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279620    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279622    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279625    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279627    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279630    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279632    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279648    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279651    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279655    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279658    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279661    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279664    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279668    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279671    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279673    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279680    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279683    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.284749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279686    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279688    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279691    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.279693    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280422    2579 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280431    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280438    2579 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280443    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280448    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280451    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280456    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280460    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280463    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280466    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280470    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280473    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280477    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280480    2579 flags.go:64] FLAG: --cgroup-root=""
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280483    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280486    2579 flags.go:64] FLAG: --client-ca-file=""
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280489    2579 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280492    2579 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280496    2579 flags.go:64] FLAG: --cluster-domain=""
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280499    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 20:03:27.285176 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280502    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280505    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280508    2579 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280513    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280516    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280519    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280524    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280528    2579 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280533    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280536    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280540    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280543    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280547    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280550    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280553    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280556    2579 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280559    2579 flags.go:64] FLAG: --enable-server="true"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280562    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280567    2579 flags.go:64] FLAG: --event-burst="100"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280570    2579 flags.go:64] FLAG: --event-qps="50"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280573    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280576    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280579    2579 flags.go:64] FLAG: --eviction-hard=""
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280584    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 20:03:27.285746 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280587    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280590    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280593    2579 flags.go:64] FLAG: --eviction-soft=""
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280596    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280599    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280602    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280605    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280608    2579 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280611    2579 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280614    2579 flags.go:64] FLAG: --feature-gates=""
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280619    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280622    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280626    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280629    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280633    2579 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280651    2579 flags.go:64] FLAG: --help="false"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280655    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280659    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280662    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280665    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280669    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280672    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280675    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 20:03:27.286319 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280678    2579 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280681    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280684    2579 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280687    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280690    2579 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280693    2579 flags.go:64] FLAG: --kube-reserved=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280696    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280699    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280702    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280705    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280708    2579 flags.go:64] FLAG: --lock-file=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280711    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280714    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280717    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280723    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280726    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280729    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280732    2579 flags.go:64] FLAG: --logging-format="text"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280735    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280739    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280742    2579 flags.go:64] FLAG: --manifest-url=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280745    2579 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280749    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280754    2579 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280758    2579 flags.go:64] FLAG: --max-pods="110"
Oct 30 20:03:27.286942 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280761    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280764    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280768    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280771    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280774    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280777    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280779    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280787    2579 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280790    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280793    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280796    2579 flags.go:64] FLAG: --pod-cidr=""
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280799    2579 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280802    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280805    2579 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280808    2579 flags.go:64] FLAG: --port="10250"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280811    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280814    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0dade436e37cb95e7"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280818    2579 flags.go:64] FLAG: --qos-reserved=""
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280821    2579 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280823    2579 flags.go:64] FLAG: --register-node="true"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280826    2579 flags.go:64] FLAG: --register-with-taints=""
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280831    2579 flags.go:64] FLAG: --registry-burst="10"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280834    2579 flags.go:64] FLAG: --registry-qps="5"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280837    2579 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280840    2579 flags.go:64] FLAG: --reserved-memory=""
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280844    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 20:03:27.287526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280847    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280850    2579 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280853    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280856    2579 flags.go:64] FLAG: --runonce="false"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280860    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280863    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280866    2579 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280869    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280872    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280876    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280880    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280883    2579 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280886    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280889    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280892    2579 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280895    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280898    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280901    2579 flags.go:64] FLAG: --system-cgroups=""
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280905    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280911    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280914    2579 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280917    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280922    2579 flags.go:64] FLAG: --tls-min-version=""
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280925    2579 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280928    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 20:03:27.288157 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280931    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280933    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280937    2579 flags.go:64] FLAG: --v="2"
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280941    2579 flags.go:64] FLAG: --version="false"
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280945    2579 flags.go:64] FLAG: --vmodule=""
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280949    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.280953    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281100    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281104    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281107    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281109    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281113    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281115    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281118    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281121    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281123    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281126    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281129    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281131    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281134    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281136    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.288749 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281139    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281141    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281144    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281148    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281153    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281156    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281159    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281162    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281165    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281168    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281170    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281173    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281175    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281178    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281181    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281183    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281186    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281188    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281191    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281193    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.289241 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281196    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281198    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281201    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281203    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281206    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281209    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281211    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281214    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281217    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281220    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281222    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281224    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281227    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281229    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281232    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281235    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281239    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281242    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281244    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281247    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.289763 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281250    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281252    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281255    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281258    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281260    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281262    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281265    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281268    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281270    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281273    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281275    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281278    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281280    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281282    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281285    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281288    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281290    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281293    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281295    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.290335 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281298    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281300    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281303    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281311    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281314    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281317    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281319    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281322    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281324    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281328    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281331    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281334    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281336    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281340    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281344    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281347    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281349    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281352    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281355    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281357    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.290815 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281360    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281362    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281365    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281368    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281370    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281373    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281376    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281378    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281381    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281383    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281386    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281388    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281391    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.281419    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281461    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.291278 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281465    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281469    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281472    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281475    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281477    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281481    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281486    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281489    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281492    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281494    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281497    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281501    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281505    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281508    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281511    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281513    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281516    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281519    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281523    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281525    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.291633 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281528    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281531    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281533    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281536    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281538    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281541    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281544    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281546    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281549    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281551    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281554    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281556    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281559    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281562    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281565    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281568    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281570    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281573    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281577    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281580    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.292183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281582    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281585    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281587    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281590    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281592    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281595    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281597    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281600    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281602    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281605    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281609    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281611    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281614    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281616    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281619    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281622    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281624    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281627    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281629    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.292664 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281632    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281648    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281653    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281657    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281661    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281665    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281669    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281673    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281677    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281680    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281683    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281687    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281689    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281692    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281695    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281697    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281701    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281703    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281706    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.293107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281709    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281711    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281716    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281719    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281723    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281726    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281729    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281732    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281734    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281737    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281740    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281742    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281745    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281748    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281750    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281752    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281755    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281757    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281760    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.293571 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281763    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281765    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281768    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281770    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281775    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281779    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281782    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.281784    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.289233    2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.289251    2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.289836    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.289842    2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289902    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289908    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289911    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289914    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289917    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289920    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289922    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.294059 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289925    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289928    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289931    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289933    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289936    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289938    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289941    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289944    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289946    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289949    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289951    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289956    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289960    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289963    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289966    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289968    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289972    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289974    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289977    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.294506 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289979    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289982    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289985    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289994    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.289997    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290000    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290002    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290005    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290008    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290011    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290013    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290016    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290018    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290021    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290024    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290026    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290029    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290031    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290034    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.294960 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290036    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290039    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290041    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290044    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290047    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290049    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290052    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290054    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290058    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290062    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290066    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290069    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290072    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290075    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290078    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290081    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290083    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290091    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290094    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290097    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.295398 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290100    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290102    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290105    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290107    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290110    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290113    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290115    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290118    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290120    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290122    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290125    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290127    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290130    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290132    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290135    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290138    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290140    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290143    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290146    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.295955 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290148    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290151    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290154    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290156    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290159    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290162    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290164    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290167    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290169    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290172    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290174    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290182    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290185    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290188    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290190    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290193    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290195    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290198    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290200    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290203    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.296410 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290205    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290207    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.290237    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290431    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290436    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290438    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290441    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290444    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290448    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290452    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290455    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290458    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290461    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290463    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290466    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.296888 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290468    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290471    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290473    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290476    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290478    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290481    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290484    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290486    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290488    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290498    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290501    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290503    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290506    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290510    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290514    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290517    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290520    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290523    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290525    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.297328 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290528    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290531    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290533    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290536    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290538    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290541    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290543    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290546    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290548    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290551    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290553    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290556    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290559    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290561    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290564    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290567    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290569    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290572    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290575    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290577    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.297828 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290580    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290582    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290584    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290592    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290595    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290598    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290600    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290603    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290606    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290608    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290611    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290613    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290616    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290618    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290621    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290623    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290626    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290628    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290630    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290633    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.298515 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290652    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290656    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290658    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290661    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290663    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290666    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290668    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290671    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290674    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290676    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290679    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290682    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290684    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290687    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290689    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290692    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290700    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290703    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290705    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.299183 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290708    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290710    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290713    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290715    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290718    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290720    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290723    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290725    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290728    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290730    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290733    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290735    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290738    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290740    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290744    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290746    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.299626 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.290770    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290809    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290813    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290816    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290819    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290822    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290824    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290827    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290830    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290832    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290835    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290837    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290840    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290842    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290850    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290853    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290856    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290858    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290861    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:03:27.300017 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290863    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290866    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290868    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290873    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290876    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290878    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290881    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290883    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290886    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290888    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290892    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290896    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290899    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290901    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290904    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290906    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290909    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290911    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290914    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:03:27.300501 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290916    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290919    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290921    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290924    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290926    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290929    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290931    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290933    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290936    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290944    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290947    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290951    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290954    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290958    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290960    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290963    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290967    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290970    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290973    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290976    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:03:27.301104 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290978    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290981    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290983    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290986    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290988    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290991    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290993    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290996    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.290998    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291001    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291004    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291006    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291009    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291011    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291014    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291016    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291019    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291021    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291024    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291026    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:03:27.301612 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291029    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291031    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291039    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291042    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291045    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291047    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291050    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291052    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291056    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291059    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291061    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291064    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291066    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291069    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291071    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291073    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291076    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291080    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291082    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:03:27.302084 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291085    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291087    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291090    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291092    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291095    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291097    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291100    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291102    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291104    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:27.291107    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.291185    2579 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.293404    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.294223    2579 server.go:1014] "Starting client certificate rotation"
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.294327    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.294820    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.296107    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:03:27.302525 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.301507    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:03:27.312458 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.312442    2579 log.go:25] "Validated CRI v1 runtime API"
Oct 30 20:03:27.315556 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.315541    2579 log.go:25] "Validated CRI v1 image API"
Oct 30 20:03:27.316513 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.316492    2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 20:03:27.320667 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.320632    2579 fs.go:135] Filesystem UUIDs: map[02667abc-60a6-46d9-b3be-b2d61d786408:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e2e521f0-2ae4-42d4-a14d-b482c126cc6b:/dev/nvme0n1p4]
Oct 30 20:03:27.320731 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.320666    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 20:03:27.325930 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.325816    2579 manager.go:217] Machine: {Timestamp:2026-10-30 20:03:27.324753137 +0000 UTC m=+0.294106892 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102331 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f965da3a7fedd028695de741d739c SystemUUID:ec2f965d-a3a7-fedd-0286-95de741d739c BootID:ff2ab5e3-9843-4f88-8a75-3c5668d0a397 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ef:54:8c:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ef:54:8c:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:1b:38:dc:60:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 20:03:27.326535 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.326525    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 20:03:27.326619 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.326608    2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 20:03:27.327752 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.327728    2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 20:03:27.327904 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.327753    2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-14.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 20:03:27.327953 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.327906    2579 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 20:03:27.327953 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.327914    2579 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 20:03:27.327953 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.327940    2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:03:27.328029 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.327954    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:03:27.328713 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.328702    2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 20:03:27.332460 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.332448    2579 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 20:03:27.332571 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.332563    2579 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 20:03:27.333276 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.333267    2579 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 20:03:27.333305 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.333279    2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 20:03:27.333305 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.333289    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 20:03:27.333305 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.333298    2579 kubelet.go:405] "Adding apiserver pod source"
Oct 30 20:03:27.333413 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.333308    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 20:03:27.333758 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.333745    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:03:27.335752 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.335733    2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 20:03:27.342348 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.342328    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:03:27.343175 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343160    2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 20:03:27.343251 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343192    2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 20:03:27.343358 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343347    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 20:03:27.343391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343364    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 20:03:27.343391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343370    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 20:03:27.343391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343375    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 20:03:27.343391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343381    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 20:03:27.343391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343389    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 20:03:27.343735 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343715    2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 20:03:27.343865 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343856    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 20:03:27.343901 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343867    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 20:03:27.343901 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343875    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 20:03:27.343901 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343881    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 20:03:27.343901 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343890    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 20:03:27.343901 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.343899    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 20:03:27.344427 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.344417    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 20:03:27.344427 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.344426    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 20:03:27.348401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.348388    2579 server.go:1263] "Started kubelet"
Oct 30 20:03:27.348504 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.348483    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-14.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:03:27.348578 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.348507    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 20:03:27.348630 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.348590    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 20:03:27.348743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.348704    2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 20:03:27.349208 ip-10-0-129-14 systemd[1]: Started Kubernetes Kubelet.
Oct 30 20:03:27.350509 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.350488    2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 20:03:27.351227 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.351212    2579 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 20:03:27.354485 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.353944    2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-14.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 20:03:27.354485 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.354163    2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 20:03:27.356433 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.356415    2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 20:03:27.357296 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.357277    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 20:03:27.357695 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.357677    2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 20:03:27.357783 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.357762    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.358273 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358254    2579 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 20:03:27.358273 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358274    2579 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 20:03:27.358402 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358262    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 20:03:27.358442 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358419    2579 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 20:03:27.358442 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358427    2579 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 20:03:27.358628 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358610    2579 factory.go:55] Registering systemd factory
Oct 30 20:03:27.358728 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358671    2579 factory.go:223] Registration of the systemd container factory successfully
Oct 30 20:03:27.358948 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358935    2579 factory.go:153] Registering CRI-O factory
Oct 30 20:03:27.359025 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.358954    2579 factory.go:223] Registration of the crio container factory successfully
Oct 30 20:03:27.359025 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.359021    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 20:03:27.359135 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.359054    2579 factory.go:103] Registering Raw factory
Oct 30 20:03:27.359135 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.359069    2579 manager.go:1199] Started watching for new ooms in manager
Oct 30 20:03:27.359939 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.359925    2579 manager.go:319] Starting recovery of all containers
Oct 30 20:03:27.365515 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.365465    2579 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-14.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 20:03:27.366563 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.365551    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-14.us-east-2.compute.internal.18e367416e8580a9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-14.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-14.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:03:27.348367529 +0000 UTC m=+0.317721304,LastTimestamp:2026-10-30 20:03:27.348367529 +0000 UTC m=+0.317721304,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-14.us-east-2.compute.internal,}"
Oct 30 20:03:27.370052 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.369843    2579 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 20:03:27.372705 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.372664    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 20:03:27.372786 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.372752    2579 manager.go:324] Recovery completed
Oct 30 20:03:27.377584 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.377569    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.379931 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.379913    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zd6h6"
Oct 30 20:03:27.380508 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.380493    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.380574 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.380522    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.380574 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.380533    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.381130 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.381115    2579 cpu_manager.go:226] "Starting" policy="none"
Oct 30 20:03:27.381130 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.381126    2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 20:03:27.381237 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.381143    2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 20:03:27.381237 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.381150    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.382511 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.382495    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.382557 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.382528    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.382557 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.382542    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.383186 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.383120    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-14.us-east-2.compute.internal.18e36741706feaf6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-14.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-14.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-14.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:03:27.380507382 +0000 UTC m=+0.349861138,LastTimestamp:2026-10-30 20:03:27.380507382 +0000 UTC m=+0.349861138,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-14.us-east-2.compute.internal,}"
Oct 30 20:03:27.383351 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.383337    2579 policy_none.go:50] "Start"
Oct 30 20:03:27.383424 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.383354    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 20:03:27.383424 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.383363    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 20:03:27.385357 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.385346    2579 policy_none.go:44] "Start"
Oct 30 20:03:27.394504 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.394486    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zd6h6"
Oct 30 20:03:27.397173 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.397076    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-14.us-east-2.compute.internal.18e367417070368d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-14.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-14.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-14.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:03:27.380526733 +0000 UTC m=+0.349880492,LastTimestamp:2026-10-30 20:03:27.380526733 +0000 UTC m=+0.349880492,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-14.us-east-2.compute.internal,}"
Oct 30 20:03:27.410607 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.410569    2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 20:03:27.414728 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.414712    2579 manager.go:341] "Starting Device Plugin manager"
Oct 30 20:03:27.414819 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.414742    2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 20:03:27.414819 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.414751    2579 server.go:87] "Starting device plugin registration server"
Oct 30 20:03:27.414975 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.414963    2579 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 20:03:27.415017 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.414979    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 20:03:27.415267 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.415084    2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 20:03:27.415267 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.415177    2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 20:03:27.415267 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.415186    2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 20:03:27.415677 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.415626    2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 20:03:27.415759 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.415720    2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.515897 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.515861    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.520449 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.520430    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.520552 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.520470    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.520552 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.520486    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.520552 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.520530    2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.529341 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.529299    2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.529341 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.529323    2579 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-14.us-east-2.compute.internal\": node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.559517 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.559496    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.659808 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.659771    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.670231 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.670208    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 20:03:27.670231 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.670235    2579 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 20:03:27.670368 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.670254    2579 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 20:03:27.671244 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.670286    2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 20:03:27.677438 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.677423    2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:03:27.760736 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.760707    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.771919 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.771881    2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"]
Oct 30 20:03:27.771993 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.771963    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.773232 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.773213    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.773322 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.773241    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.773322 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.773252    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.775591 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.775578    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.775757 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.775739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.775808 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.775772    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.777569 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.777552    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.777625 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.777579    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.777625 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.777589    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.777710 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.777559    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.777710 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.777662    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.777710 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.777673    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.779811 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.779776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.779811 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.779802    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:27.780592 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.780574    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:27.780689 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.780604    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:27.780689 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.780616    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:27.805131 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.805113    2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.807898 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.807879    2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.861519 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.861494    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.861625 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.861524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.861625 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.861548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.861625 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.861565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfb87cbb02f3462c0bb6e1279ac3fe5a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"bfb87cbb02f3462c0bb6e1279ac3fe5a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.961896 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:27.961864    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:27.962033 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.961917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.962033 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.961943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.962033 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.961968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfb87cbb02f3462c0bb6e1279ac3fe5a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"bfb87cbb02f3462c0bb6e1279ac3fe5a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.962033 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.961998    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.962033 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.962003    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfb87cbb02f3462c0bb6e1279ac3fe5a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"bfb87cbb02f3462c0bb6e1279ac3fe5a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:27.962183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:27.962035    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:28.062421 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:28.062362    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:28.108557 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:28.108521    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:28.110603 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:28.110588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:28.162764 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:28.162597    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:28.263021 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:28.262986    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:28.294198 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:28.294159    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 20:03:28.294677 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:28.294359    2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 20:03:28.363118 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:28.363043    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:28.399651 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:28.399570    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:58:28 +0000 UTC" deadline="2028-06-16 12:18:55.327110772 +0000 UTC"
Oct 30 20:03:28.399651 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:28.399654    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14272h15m26.927460291s"
Oct 30 20:03:29.523558 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:29.523531    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:29.580725 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.580697    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:03:29.581701 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.581683    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:03:29.581806 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.581721    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:03:29.581806 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.581736    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:03:29.592725 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.592706    2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:03:29.624064 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:29.624041    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:03:29.638915 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.638897    2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:03:29.658309 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.658291    2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:03:29.718431 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.718395    2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:29.731367 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.731338    2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:03:29.732388 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.732370    2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:03:29.742566 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.742549    2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:03:29.815565 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:29.815536    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb87cbb02f3462c0bb6e1279ac3fe5a.slice/crio-ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea WatchSource:0}: Error finding container ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea: Status 404 returned error can't find the container with id ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea
Oct 30 20:03:29.826098 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:29.826078    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:03:29.833107 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:29.833082    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e91bd056ef60d099398ca83c4eeeeab.slice/crio-743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac WatchSource:0}: Error finding container 743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac: Status 404 returned error can't find the container with id 743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac
Oct 30 20:03:30.395156 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.394867    2579 apiserver.go:52] "Watching apiserver"
Oct 30 20:03:30.404868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.404831    2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 20:03:30.405242 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.405218    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-c9kfm","openshift-network-diagnostics/network-check-target-zv26m","openshift-network-operator/iptables-alerter-6whf2","openshift-image-registry/node-ca-zhts8","openshift-multus/multus-additional-cni-plugins-49thc","openshift-ovn-kubernetes/ovnkube-node-jnfz5","kube-system/global-pull-secret-syncer-sgpdx","kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal","kube-system/konnectivity-agent-9jh8v","openshift-cluster-node-tuning-operator/tuned-fbmvs","openshift-dns/node-resolver-mzzv9","openshift-multus/multus-8lgs5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"]
Oct 30 20:03:30.407781 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.407758    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.407878 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.407843    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:30.409935 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.409911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.413662 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.413640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.416567 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.416547    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.416662 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.416635    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 20:03:30.416764 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.416749    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hms2c\""
Oct 30 20:03:30.416854 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.416833    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.417323 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.417302    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.417428 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.417412    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 20:03:30.417654 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.417635    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 20:03:30.417738 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.417723    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rlckg\""
Oct 30 20:03:30.417807 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.417795    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.419034 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.419014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.421385 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.421364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.429516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-registration-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.429594 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-sys-fs\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.429594 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de259d4b-08fb-48bb-9aad-0a381205090f-iptables-alerter-script\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.429662 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5ddf8fa-c589-4ef9-8299-83ac15845b10-host\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.429662 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.429738 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-device-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.429738 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.429828 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de259d4b-08fb-48bb-9aad-0a381205090f-host-slash\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.429828 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429799    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7wjj\" (UniqueName: \"kubernetes.io/projected/de259d4b-08fb-48bb-9aad-0a381205090f-kube-api-access-s7wjj\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.429893 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5ddf8fa-c589-4ef9-8299-83ac15845b10-serviceca\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.429945 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.429997 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.429961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-socket-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.430045 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.430032    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdc9g\" (UniqueName: \"kubernetes.io/projected/8bdc9936-09aa-4583-95e7-499428c2b9b0-kube-api-access-sdc9g\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.430100 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.430078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-dbus\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.430196 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.430181    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.431039 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.431022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.431184 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.431050    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 20:03:30.431488 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.431469    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc8vv\" (UniqueName: \"kubernetes.io/projected/b5ddf8fa-c589-4ef9-8299-83ac15845b10-kube-api-access-pc8vv\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.431581 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.431504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-kubelet-config\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.433196 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.433177    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:30.433289 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.433265    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:30.435704 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.435685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:30.435799 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.435747    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:30.438930 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.438140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.440535 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.440517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.442843 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.442824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.447673 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.447472    2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 20:03:30.460032 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.459892    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-drs5z"
Oct 30 20:03:30.472494 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.472470    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-drs5z"
Oct 30 20:03:30.498937 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.497584    2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:03:30.519055 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519026    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 20:03:30.519348 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519317    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.519453 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519439    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 20:03:30.519516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519486    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 20:03:30.519570 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519519    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.519570 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519561    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 20:03:30.519666 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519645    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 20:03:30.519877 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.519862    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.525747    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.525827    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-26gph\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.525896    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.525977    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-84sqw\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526074    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rnw2v\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526157    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-phqsm\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526294    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rx62b\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526355    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526517    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526614    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526657    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526740    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526761    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526821    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-cfnmr\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526849    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526889    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526943    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.526992    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 20:03:30.528743 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.527058    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:03:30.529860 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.529768    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hntbg\""
Oct 30 20:03:30.531736 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531695    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-netns\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.531736 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:30.531917 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531752    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysconfig\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.531917 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-tmp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.531917 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-registration-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.531917 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eab05532-1d94-4218-bef2-a7f43bfd25e8-agent-certs\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.531917 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-conf-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531951    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-registration-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.531971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-systemd\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.532100    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-etc-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.532183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-script-lib\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.532220    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:31.032193357 +0000 UTC m=+2.942067760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-conf\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-log-socket\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdc9g\" (UniqueName: \"kubernetes.io/projected/8bdc9936-09aa-4583-95e7-499428c2b9b0-kube-api-access-sdc9g\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-dbus\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cni-binary-copy\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.532516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-netns\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-modprobe-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-kubelet-config\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532638    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532648    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-dbus\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532656    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-kubelet-config\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-var-lib-kubelet\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532734    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-host\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-system-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-multus\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-multus-certs\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.532868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tr6l\" (UniqueName: \"kubernetes.io/projected/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-kube-api-access-7tr6l\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532876    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-ovn\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-sys\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.532988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5ddf8fa-c589-4ef9-8299-83ac15845b10-serviceca\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cnibin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-slash\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-systemd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-run\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efb380a-6530-4db0-88ec-2d59891344f1-hosts-file\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk2l2\" (UniqueName: \"kubernetes.io/projected/5efb380a-6530-4db0-88ec-2d59891344f1-kube-api-access-dk2l2\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-socket-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-kubelet\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533370    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-daemon-config\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-kubelet\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovn-node-metrics-cert\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgrsp\" (UniqueName: \"kubernetes.io/projected/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-kube-api-access-tgrsp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.533510 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-bin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzlpm\" (UniqueName: \"kubernetes.io/projected/cdaa2197-530d-48e5-899c-5ccc58f4016a-kube-api-access-zzlpm\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533508    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5ddf8fa-c589-4ef9-8299-83ac15845b10-serviceca\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q59hq\" (UniqueName: \"kubernetes.io/projected/4ea62f92-610f-4211-9153-474cd90cadaa-kube-api-access-q59hq\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-lib-modules\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-tuned\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-k8s-cni-cncf-io\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533654    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-socket-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de259d4b-08fb-48bb-9aad-0a381205090f-iptables-alerter-script\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eab05532-1d94-4218-bef2-a7f43bfd25e8-konnectivity-ca\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-systemd-units\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-config\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-env-overrides\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-device-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7wjj\" (UniqueName: \"kubernetes.io/projected/de259d4b-08fb-48bb-9aad-0a381205090f-kube-api-access-s7wjj\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.534347 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-hostroot\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.533954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-var-lib-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-netd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-node-log\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-socket-dir-parent\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534743    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de259d4b-08fb-48bb-9aad-0a381205090f-iptables-alerter-script\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534218    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-device-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-bin\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc8vv\" (UniqueName: \"kubernetes.io/projected/b5ddf8fa-c589-4ef9-8299-83ac15845b10-kube-api-access-pc8vv\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8gkt\" (UniqueName: \"kubernetes.io/projected/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-kube-api-access-t8gkt\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.534987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-sys-fs\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5ddf8fa-c589-4ef9-8299-83ac15845b10-host\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.535112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-etc-kubernetes\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535091    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cnibin\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-kubernetes\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efb380a-6530-4db0-88ec-2d59891344f1-tmp-dir\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535180    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-sys-fs\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de259d4b-08fb-48bb-9aad-0a381205090f-host-slash\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535455    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5ddf8fa-c589-4ef9-8299-83ac15845b10-host\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535560    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535659    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.535867 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.535695    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de259d4b-08fb-48bb-9aad-0a381205090f-host-slash\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.544265 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.544218    2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 20:03:30.548847 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.548825    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7wjj\" (UniqueName: \"kubernetes.io/projected/de259d4b-08fb-48bb-9aad-0a381205090f-kube-api-access-s7wjj\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.548968 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.548884    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdc9g\" (UniqueName: \"kubernetes.io/projected/8bdc9936-09aa-4583-95e7-499428c2b9b0-kube-api-access-sdc9g\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.549279 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.549257    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc8vv\" (UniqueName: \"kubernetes.io/projected/b5ddf8fa-c589-4ef9-8299-83ac15845b10-kube-api-access-pc8vv\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.636566 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-etc-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.636762 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636598    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-etc-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.636762 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-script-lib\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.636762 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-conf\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.636762 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-log-socket\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.636762 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cni-binary-copy\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.636762 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-netns\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636790    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-modprobe-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-var-lib-kubelet\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-host\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-system-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-multus\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-multus-certs\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tr6l\" (UniqueName: \"kubernetes.io/projected/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-kube-api-access-7tr6l\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-ovn\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-sys\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cnibin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.636989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-slash\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-systemd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-run\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efb380a-6530-4db0-88ec-2d59891344f1-hosts-file\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk2l2\" (UniqueName: \"kubernetes.io/projected/5efb380a-6530-4db0-88ec-2d59891344f1-kube-api-access-dk2l2\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-kubelet\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-daemon-config\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-kubelet\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovn-node-metrics-cert\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrsp\" (UniqueName: \"kubernetes.io/projected/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-kube-api-access-tgrsp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-bin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637272    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-script-lib\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzlpm\" (UniqueName: \"kubernetes.io/projected/cdaa2197-530d-48e5-899c-5ccc58f4016a-kube-api-access-zzlpm\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q59hq\" (UniqueName: \"kubernetes.io/projected/4ea62f92-610f-4211-9153-474cd90cadaa-kube-api-access-q59hq\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-lib-modules\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-tuned\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637429    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637449    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.637748 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-k8s-cni-cncf-io\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eab05532-1d94-4218-bef2-a7f43bfd25e8-konnectivity-ca\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-systemd-units\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-config\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637554    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-conf\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-env-overrides\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-hostroot\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-var-lib-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-netd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-node-log\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-socket-dir-parent\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-bin\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8gkt\" (UniqueName: \"kubernetes.io/projected/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-kube-api-access-t8gkt\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.637949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:30.638543 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-etc-kubernetes\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cnibin\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638062    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-sys\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-kubernetes\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efb380a-6530-4db0-88ec-2d59891344f1-tmp-dir\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638136    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cnibin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638179    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-slash\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638215    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-systemd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-netns\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638259    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-run\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysconfig\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-tmp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eab05532-1d94-4218-bef2-a7f43bfd25e8-agent-certs\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-conf-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.639391 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-systemd\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638527    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-log-socket\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638543    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-systemd\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638588    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efb380a-6530-4db0-88ec-2d59891344f1-hosts-file\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638750    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-kubelet\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639074    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cni-binary-copy\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639160    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-netns\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639211    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639244    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-daemon-config\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639285    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-kubelet\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639327    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-modprobe-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639393    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639446    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-var-lib-kubelet\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639493    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-k8s-cni-cncf-io\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639499    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-host\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639566    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-system-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639617    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-multus\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639651    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-multus-certs\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.640278 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639721    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-bin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639925    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-ovn\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.639985    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-hostroot\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640094    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-lib-modules\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640555    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640797    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640843    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-var-lib-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640890    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-netd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640935    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-node-log\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.641243 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.640983    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-socket-dir-parent\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.641657 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.641451    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.641657 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.641511    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-bin\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.641809 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.641791    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:30.641887 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.641849    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:03:31.141832662 +0000 UTC m=+3.051707050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:30.641887 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.641875    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovn-node-metrics-cert\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.642001 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.641913    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.642050 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642030    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.642102 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642088    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.642181 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642153    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-netns\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.642364 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642336    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-etc-kubernetes\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.642364 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.638023    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-env-overrides\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.642520 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642403    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cnibin\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.642520 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642423    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-systemd-units\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.642520 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642450    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-kubernetes\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.642520 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642488    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysconfig\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.642520 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642508    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-conf-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.642801 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.642782    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efb380a-6530-4db0-88ec-2d59891344f1-tmp-dir\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.643091 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.643045    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eab05532-1d94-4218-bef2-a7f43bfd25e8-konnectivity-ca\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.643367 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.643344    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-tuned\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.643624 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.643601    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.643624 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.643614    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-config\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.645853 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.645802    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-tmp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.645930 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.645908    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eab05532-1d94-4218-bef2-a7f43bfd25e8-agent-certs\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.653205 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.653183    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:03:30.653302 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.653211    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:03:30.653302 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.653225    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:30.653427 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:30.653337    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:03:31.153317738 +0000 UTC m=+3.063192142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:30.656337 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.656309    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8gkt\" (UniqueName: \"kubernetes.io/projected/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-kube-api-access-t8gkt\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.656467 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.656446    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrsp\" (UniqueName: \"kubernetes.io/projected/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-kube-api-access-tgrsp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.657231 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.657211    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q59hq\" (UniqueName: \"kubernetes.io/projected/4ea62f92-610f-4211-9153-474cd90cadaa-kube-api-access-q59hq\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:30.658046 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.657786    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzlpm\" (UniqueName: \"kubernetes.io/projected/cdaa2197-530d-48e5-899c-5ccc58f4016a-kube-api-access-zzlpm\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.658046 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.657995    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tr6l\" (UniqueName: \"kubernetes.io/projected/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-kube-api-access-7tr6l\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.659671 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.659648    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk2l2\" (UniqueName: \"kubernetes.io/projected/5efb380a-6530-4db0-88ec-2d59891344f1-kube-api-access-dk2l2\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.731953 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.731922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:03:30.734586 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.734532    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"1e91bd056ef60d099398ca83c4eeeeab","Type":"ContainerStarted","Data":"743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac"}
Oct 30 20:03:30.736225 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.736200    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"bfb87cbb02f3462c0bb6e1279ac3fe5a","Type":"ContainerStarted","Data":"ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea"}
Oct 30 20:03:30.741553 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.741531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:03:30.744223 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:30.744184    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdc9936_09aa_4583_95e7_499428c2b9b0.slice/crio-bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e WatchSource:0}: Error finding container bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e: Status 404 returned error can't find the container with id bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e
Oct 30 20:03:30.750909 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.750890    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:03:30.764232 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:30.764203    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ddf8fa_c589_4ef9_8299_83ac15845b10.slice/crio-084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644 WatchSource:0}: Error finding container 084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644: Status 404 returned error can't find the container with id 084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644
Oct 30 20:03:30.778295 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.778270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:03:30.779396 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.779377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:30.790436 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:30.790384    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba6f137_f70b_41a4_9e2b_871bb10b4f07.slice/crio-5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70 WatchSource:0}: Error finding container 5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70: Status 404 returned error can't find the container with id 5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70
Oct 30 20:03:30.791828 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.791765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:03:30.793865 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:30.793838    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab05532_1d94_4218_bef2_a7f43bfd25e8.slice/crio-969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454 WatchSource:0}: Error finding container 969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454: Status 404 returned error can't find the container with id 969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454
Oct 30 20:03:30.795875 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.795065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:03:30.799247 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.798671    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:30.802955 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:30.801905    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lgs5"
Oct 30 20:03:30.811882 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:30.811847    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efb380a_6530_4db0_88ec_2d59891344f1.slice/crio-e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868 WatchSource:0}: Error finding container e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868: Status 404 returned error can't find the container with id e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868
Oct 30 20:03:30.820929 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:03:30.820850    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de1609a_ab4c_448c_bc7f_fcdbcc70ff41.slice/crio-028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89 WatchSource:0}: Error finding container 028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89: Status 404 returned error can't find the container with id 028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89
Oct 30 20:03:31.042680 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.042611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:31.042853 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.042780    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:31.042853 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.042837    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:32.042824311 +0000 UTC m=+3.952698706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:31.143789 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.143747    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:31.143940 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.143914    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:31.144016 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.143996    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:03:32.143977078 +0000 UTC m=+4.053851483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:31.244363 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.244333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:31.244532 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.244511    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:03:31.244577 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.244540    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:03:31.244577 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.244552    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:31.244652 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.244605    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:03:32.244593029 +0000 UTC m=+4.154467417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:31.476363 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.476261    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:58:30 +0000 UTC" deadline="2028-04-12 21:27:27.033237903 +0000 UTC"
Oct 30 20:03:31.476363 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.476320    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12721h23m55.556921979s"
Oct 30 20:03:31.730702 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.730155    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:31.730702 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.730206    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:31.730702 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.730311    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:31.730702 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:31.730411    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:31.749772 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.749599    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lgs5" event={"ID":"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41","Type":"ContainerStarted","Data":"028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89"}
Oct 30 20:03:31.766072 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.766011    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fbmvs" event={"ID":"f4ce54b1-7691-41ff-9a3e-56b99c04ced4","Type":"ContainerStarted","Data":"a77cd53b79ba8fd5a680d6e2c31b2a6ea073d8101ac5912be2e3288f97111796"}
Oct 30 20:03:31.772495 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.772441    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mzzv9" event={"ID":"5efb380a-6530-4db0-88ec-2d59891344f1","Type":"ContainerStarted","Data":"e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868"}
Oct 30 20:03:31.777828 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.777801    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zhts8" event={"ID":"b5ddf8fa-c589-4ef9-8299-83ac15845b10","Type":"ContainerStarted","Data":"084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644"}
Oct 30 20:03:31.783935 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.783908    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn" event={"ID":"8bdc9936-09aa-4583-95e7-499428c2b9b0","Type":"ContainerStarted","Data":"bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e"}
Oct 30 20:03:31.788999 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.788970    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"a40bd710d363b612693bc3c34aa642eaae991f884783c59ef78f254aecbddd48"}
Oct 30 20:03:31.802233 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.802208    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jh8v" event={"ID":"eab05532-1d94-4218-bef2-a7f43bfd25e8","Type":"ContainerStarted","Data":"969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454"}
Oct 30 20:03:31.809000 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.808962    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerStarted","Data":"5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70"}
Oct 30 20:03:31.821711 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:31.821682    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6whf2" event={"ID":"de259d4b-08fb-48bb-9aad-0a381205090f","Type":"ContainerStarted","Data":"5ce83785c43575b988dfc59ee502446025dde26b6a67fe37d3a4e1784bc14e53"}
Oct 30 20:03:32.050393 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.050367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:32.050559 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.050544    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:32.050646 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.050633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:34.05061419 +0000 UTC m=+5.960488593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:32.152238 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.152207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:32.152385 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.152363    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:32.152503 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.152428    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:03:34.152409807 +0000 UTC m=+6.062284210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:32.253445 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.252733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:32.253445 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.252912    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:03:32.253445 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.252932    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:03:32.253445 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.252944    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:32.253445 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.253004    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:03:34.252986166 +0000 UTC m=+6.162860573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:32.476649 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.476604    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:58:30 +0000 UTC" deadline="2028-07-11 13:43:04.306566256 +0000 UTC"
Oct 30 20:03:32.476649 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.476648    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14873h39m31.829923311s"
Oct 30 20:03:32.733327 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.732682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:32.733327 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:32.732806    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:32.834523 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.834362    2579 generic.go:356] "Generic (PLEG): container finished" podID="1e91bd056ef60d099398ca83c4eeeeab" containerID="4aad6c6a185c7ede00a6644377cbeee5f226de796ed3e136fba74e66da506540" exitCode=0
Oct 30 20:03:32.834523 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.834458    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"1e91bd056ef60d099398ca83c4eeeeab","Type":"ContainerDied","Data":"4aad6c6a185c7ede00a6644377cbeee5f226de796ed3e136fba74e66da506540"}
Oct 30 20:03:32.845303 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.845278    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"bfb87cbb02f3462c0bb6e1279ac3fe5a","Type":"ContainerStarted","Data":"e7abda52eed75aae252331091b5b802aad74d0d2165cb174d4e1989888cba360"}
Oct 30 20:03:32.909041 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:32.908983    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal" podStartSLOduration=3.908959474 podStartE2EDuration="3.908959474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:32.908384406 +0000 UTC m=+4.818258819" watchObservedRunningTime="2026-10-30 20:03:32.908959474 +0000 UTC m=+4.818833885"
Oct 30 20:03:33.730845 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:33.730817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:33.731245 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:33.730941    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:33.731394 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:33.731333    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:33.731465 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:33.731438    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:33.854761 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:33.854713    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"1e91bd056ef60d099398ca83c4eeeeab","Type":"ContainerStarted","Data":"29e6574cbf806c0e8419c3137c02924f7e3edba1f88ac0f7ee5c0f7a2bc8b129"}
Oct 30 20:03:33.875091 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:33.875038    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" podStartSLOduration=4.875018301 podStartE2EDuration="4.875018301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:33.874721315 +0000 UTC m=+5.784595756" watchObservedRunningTime="2026-10-30 20:03:33.875018301 +0000 UTC m=+5.784892712"
Oct 30 20:03:34.069099 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:34.068419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:34.069099 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.068589    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:34.069099 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.068653    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:38.068633452 +0000 UTC m=+9.978507849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:34.169671 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:34.169058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:34.169671 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.169262    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:34.169671 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.169324    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:03:38.169306788 +0000 UTC m=+10.079181192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:34.270164 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:34.270026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:34.270915 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.270520    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:03:34.270915 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.270549    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:03:34.270915 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.270562    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:34.270915 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.270625    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:03:38.270607673 +0000 UTC m=+10.180482061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:34.732909 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:34.732872    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:34.733416 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:34.733004    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:34.867252 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:34.865961    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="824d7745658ac6bd7ae58d98b1eb24c73fa7162e78d253ab3dd31c24e49e1818" exitCode=0
Oct 30 20:03:34.867252 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:34.867001    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerDied","Data":"824d7745658ac6bd7ae58d98b1eb24c73fa7162e78d253ab3dd31c24e49e1818"}
Oct 30 20:03:35.730143 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:35.730087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:35.730324 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:35.730233    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:35.730819 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:35.730686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:35.730819 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:35.730805    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:35.878819 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:35.878780    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jh8v" event={"ID":"eab05532-1d94-4218-bef2-a7f43bfd25e8","Type":"ContainerStarted","Data":"f37b4f6da25edf9bc893261b486c04f21559ac4cd54fd14e259806a2ad198ada"}
Oct 30 20:03:35.908568 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:35.908513    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9jh8v" podStartSLOduration=3.869655426 podStartE2EDuration="7.908481393s" totalImagesPullingTime="4.038825967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:35.907703607 +0000 UTC m=+7.817578095" watchObservedRunningTime="2026-10-30 20:03:35.908481393 +0000 UTC m=+7.818355805"
Oct 30 20:03:36.314198 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:36.313676    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:36.314198 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:36.314132    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:36.733689 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:36.733207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:36.733689 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:36.733333    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:36.884786 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:36.884718    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zhts8" event={"ID":"b5ddf8fa-c589-4ef9-8299-83ac15845b10","Type":"ContainerStarted","Data":"82dcd900d0bb05c92c1a7bea13c70e183eb69d86179d933a6f256aff4d32667d"}
Oct 30 20:03:36.891778 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:36.891753    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn" event={"ID":"8bdc9936-09aa-4583-95e7-499428c2b9b0","Type":"ContainerStarted","Data":"1ebd9ae5fa3a12a7a5f40da158a87e8b0c0459e4f7cfd75dfd60be632d591dc1"}
Oct 30 20:03:37.698554 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:37.698483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:37.732664 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:37.730530    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:37.732664 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:37.730659    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:37.732664 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:37.730719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:37.732664 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:37.730844    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:37.802629 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:37.801415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:37.894548 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:37.894524    2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 20:03:37.950959 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:37.950906    2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 20:03:38.105366 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.105327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:38.105524 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.105500    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:38.105591 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.105582    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:46.105562285 +0000 UTC m=+18.015436688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:38.207134 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.206573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:38.207134 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.206730    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:38.207134 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.206789    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:03:46.206774676 +0000 UTC m=+18.116649077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:38.307137 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.307094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:38.307351 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.307319    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:03:38.307446 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.307354    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:03:38.307446 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.307367    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:38.307446 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.307432    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:03:46.307415385 +0000 UTC m=+18.217289775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:38.480178 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.479494    2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T20:03:37.950922891Z","UUID":"128f81cc-8856-4107-a8db-5ee6744cca9f","Handler":null,"Name":"","Endpoint":""}
Oct 30 20:03:38.481516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.481073    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 20:03:38.481516 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.481108    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 20:03:38.524228 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.524165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:38.644560 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.644443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:38.732857 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.732794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:38.733027 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:38.732993    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:38.834495 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.834362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:38.898772 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.898738    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lgs5" event={"ID":"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41","Type":"ContainerStarted","Data":"65588eeecce345e7cd556dd30adce06eae535a55e8cdf94f4d280c4270cbd7c0"}
Oct 30 20:03:38.901803 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.901401    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mzzv9" event={"ID":"5efb380a-6530-4db0-88ec-2d59891344f1","Type":"ContainerStarted","Data":"591acbb87adbfedb1933142621dff0268c5a1d507446e7fbd7c52c9daf874915"}
Oct 30 20:03:38.906032 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.906004    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn" event={"ID":"8bdc9936-09aa-4583-95e7-499428c2b9b0","Type":"ContainerStarted","Data":"ae5984f367c6b548ec3f5a044589f4bdd87c3b9ca28745bb027b50c5e1fe3a20"}
Oct 30 20:03:38.923687 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.922924    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zhts8" podStartSLOduration=5.894273615 podStartE2EDuration="10.922907112s" totalImagesPullingTime="5.028633497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:36.903685313 +0000 UTC m=+8.813559736" watchObservedRunningTime="2026-10-30 20:03:38.922907112 +0000 UTC m=+10.832781535"
Oct 30 20:03:38.923687 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.923534    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-8lgs5" podStartSLOduration=3.354467366 podStartE2EDuration="10.923523644s" totalImagesPullingTime="7.569056278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:38.923287964 +0000 UTC m=+10.833162375" watchObservedRunningTime="2026-10-30 20:03:38.923523644 +0000 UTC m=+10.833398056"
Oct 30 20:03:38.944838 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.944139    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-mzzv9" podStartSLOduration=3.091980343 podStartE2EDuration="10.944109104s" totalImagesPullingTime="7.852128761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:38.94333284 +0000 UTC m=+10.853207255" watchObservedRunningTime="2026-10-30 20:03:38.944109104 +0000 UTC m=+10.853983511"
Oct 30 20:03:38.945649 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:38.945599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:39.071614 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.071535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:39.177559 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.176944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:39.303489 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.303460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:39.730611 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.730578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:39.730767 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.730630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:39.730767 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:39.730707    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:39.730897 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:39.730820    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:39.910959 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.910923    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="5eb78c901575be229432422e8c81b792c7d890f11c3dca363bc78cc9a7ab5e7b" exitCode=0
Oct 30 20:03:39.911497 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.911037    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerDied","Data":"5eb78c901575be229432422e8c81b792c7d890f11c3dca363bc78cc9a7ab5e7b"}
Oct 30 20:03:39.914274 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.914250    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn" event={"ID":"8bdc9936-09aa-4583-95e7-499428c2b9b0","Type":"ContainerStarted","Data":"93f10a1a135fc0efd3bff2dd987d17440e02ec36b7b61e8a48fd891d37827179"}
Oct 30 20:03:39.959589 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:39.959542    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn" podStartSLOduration=3.9088670519999997 podStartE2EDuration="11.959528723s" totalImagesPullingTime="8.050661671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:39.959298908 +0000 UTC m=+11.869173299" watchObservedRunningTime="2026-10-30 20:03:39.959528723 +0000 UTC m=+11.869403132"
Oct 30 20:03:40.172726 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.172632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:40.520577 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.520485    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzzv9_5efb380a-6530-4db0-88ec-2d59891344f1/dns-node-resolver/0.log"
Oct 30 20:03:40.730154 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.730127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:40.730281 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:40.730258    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:40.847608 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.847576    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:40.917359 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.917318    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fbmvs" event={"ID":"f4ce54b1-7691-41ff-9a3e-56b99c04ced4","Type":"ContainerStarted","Data":"81810ed26dcd0b6c397d4c2e4f112ba3662d99bb3beaa7e36332865267910365"}
Oct 30 20:03:40.919937 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.919907    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="045becbc89c720f021ae1ec601f7af4586a4a693ffe009d56f5a66f707649dde" exitCode=0
Oct 30 20:03:40.920058 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.919998    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerDied","Data":"045becbc89c720f021ae1ec601f7af4586a4a693ffe009d56f5a66f707649dde"}
Oct 30 20:03:40.922329 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.922298    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6whf2" event={"ID":"de259d4b-08fb-48bb-9aad-0a381205090f","Type":"ContainerStarted","Data":"ba2975380803175b35a12c60e20ba4717c31ab0f48e993e7a2e79bf1a4519115"}
Oct 30 20:03:40.938282 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.938202    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fbmvs" podStartSLOduration=3.064371856 podStartE2EDuration="12.938185483s" totalImagesPullingTime="9.873813627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:40.93807193 +0000 UTC m=+12.847946360" watchObservedRunningTime="2026-10-30 20:03:40.938185483 +0000 UTC m=+12.848059895"
Oct 30 20:03:40.981720 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.981693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:40.993439 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:40.993387    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6whf2" podStartSLOduration=4.889809001 podStartE2EDuration="12.99337055s" totalImagesPullingTime="8.103561549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:40.960908281 +0000 UTC m=+12.870782695" watchObservedRunningTime="2026-10-30 20:03:40.99337055 +0000 UTC m=+12.903244963"
Oct 30 20:03:41.109292 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.109258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:41.228263 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.228236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:41.608309 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.608208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:41.719664 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.719637    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:41.730221 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.730192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:41.730303 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.730207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:41.730336 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:41.730299    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:41.730397 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:41.730381    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:41.920769 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.920697    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhts8_b5ddf8fa-c589-4ef9-8299-83ac15845b10/node-ca/0.log"
Oct 30 20:03:41.926971 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.926943    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="007d6ecc5797bba0a63ebc82bf25bfd3a203b875b448ae19dc3e4f14553b0e03" exitCode=0
Oct 30 20:03:41.927094 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:41.927040    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerDied","Data":"007d6ecc5797bba0a63ebc82bf25bfd3a203b875b448ae19dc3e4f14553b0e03"}
Oct 30 20:03:42.360006 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:42.359974    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:42.732288 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:42.732254    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:42.732439 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:42.732361    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:42.757336 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:42.757271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:42.952784 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:42.952743    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:43.160133 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:43.160040    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:43.354934 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:43.354657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:43.466237 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:43.466212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:43.730771 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:43.730696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:43.730930 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:43.730815    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:43.730930 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:43.730863    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:43.731047 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:43.730991    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:43.946026 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:43.945997    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerStarted","Data":"0197a3e51ca3a6e60081a339a0b24ebb9eeb5427c85d65dede1f540961a75229"}
Oct 30 20:03:44.094683 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.094645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:44.556289 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.556258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:44.730394 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.730366    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:44.730524 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:44.730472    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:44.757310 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.757280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:44.950426 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.950352    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="0197a3e51ca3a6e60081a339a0b24ebb9eeb5427c85d65dede1f540961a75229" exitCode=0
Oct 30 20:03:44.950426 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.950391    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerDied","Data":"0197a3e51ca3a6e60081a339a0b24ebb9eeb5427c85d65dede1f540961a75229"}
Oct 30 20:03:44.953160 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:44.953140    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:45.153863 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.153826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:45.318649 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.318530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:45.421282 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.421257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:45.730383 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.730358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:45.730383 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.730390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:45.730605 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:45.730465    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:45.730605 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:45.730580    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:45.955614 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.955586    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="90e740d66308ef7bf5ce5e7ab45b7e88633dbf64cff204c380e56e00b847927f" exitCode=0
Oct 30 20:03:45.955776 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.955680    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerDied","Data":"90e740d66308ef7bf5ce5e7ab45b7e88633dbf64cff204c380e56e00b847927f"}
Oct 30 20:03:45.958572 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.958557    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:03:45.958909 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.958892    2579 generic.go:356] "Generic (PLEG): container finished" podID="cdaa2197-530d-48e5-899c-5ccc58f4016a" containerID="1cec20fec7905162490d7ff0b1420979b6a2d6d2b80a98eb66677e509a427f58" exitCode=1
Oct 30 20:03:45.958975 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.958917    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"7e364b87e37a9f27ba70941f8e79623049b06aeaa3759fafcb6d9b0d29a683e0"}
Oct 30 20:03:45.959012 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.958972    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"cce13c903fa2d69fd88eef54169b681c56b95322ea125fef5151d7f6efb5b3ec"}
Oct 30 20:03:45.959012 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.959004    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"48f35b44043772278dd7906643f5a08eb13cfb00b2e2e03ab931688726e1c091"}
Oct 30 20:03:45.959073 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.959034    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"186516a526960afced14922b699ec1acc014c0927ecbe0f1131fed04f4ed67f4"}
Oct 30 20:03:45.959073 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.959060    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerDied","Data":"1cec20fec7905162490d7ff0b1420979b6a2d6d2b80a98eb66677e509a427f58"}
Oct 30 20:03:45.959148 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:45.959087    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"ae1c9135f1fbe62a9e08da3c245b98ffbf3a807c46282535565366efe55a84bc"}
Oct 30 20:03:46.048132 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.048099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:46.170164 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.170138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:46.170548 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.170259    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:46.170548 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.170318    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:02.170296145 +0000 UTC m=+34.080170537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:03:46.271371 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.271318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:46.271464 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.271423    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:46.271501 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.271465    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:04:02.271453204 +0000 UTC m=+34.181327605 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:03:46.372544 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.372519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:46.372649 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.372630    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:03:46.372649 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.372649    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:03:46.372714 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.372658    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:46.372714 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.372699    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:04:02.372685108 +0000 UTC m=+34.282559498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:03:46.554041 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.553972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:46.730935 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.730907    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:46.731091 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:46.731005    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:46.756067 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.756041    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:46.954875 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.954800    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:46.964307 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.964281    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49thc" event={"ID":"6ba6f137-f70b-41a4-9e2b-871bb10b4f07","Type":"ContainerStarted","Data":"87bc0035df98a6a212a123451ae98a619bc44b0a6627667f00d617d09a1eb6c8"}
Oct 30 20:03:46.991755 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:46.991716    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-49thc" podStartSLOduration=8.460542289 podStartE2EDuration="18.991705001s" totalImagesPullingTime="10.531162712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:46.991403299 +0000 UTC m=+18.901277709" watchObservedRunningTime="2026-10-30 20:03:46.991705001 +0000 UTC m=+18.901579411"
Oct 30 20:03:47.057104 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.057083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.162169 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.162101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.355408 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.355374    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.461047 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.461015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.578933 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.578906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.700094 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.700013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.729963 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.729935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:47.729963 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.729960    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:47.730112 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:47.730039    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:47.730177 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:47.730166    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:47.810336 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.810307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.911497 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.911464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:47.968851 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.968828    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:03:47.969226 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:47.969204    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"1b0a08dcdc36c031e5e50e5d9c0c61bfd05b1b569046e3c57c7e988b28f56f3b"}
Oct 30 20:03:48.196952 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.196922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:48.298165 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.298071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:48.488611 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.488581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:48.588890 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.588817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:48.731033 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.731005    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:48.731190 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:48.731085    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:48.759194 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.759167    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:48.967808 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:48.967776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:49.153409 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.153381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:49.358701 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.358626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:49.560565 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.560537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:49.685722 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.685598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:49.730785 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.730757    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:49.730785 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.730778    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:49.730972 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:49.730864    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:49.730972 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:49.730923    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:49.976682 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.976660    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:03:49.977071 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.977049    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"629b50c9f58aa2d3cbad15e6addbd0499ae6a69d64a9a6f1bb0625b208ff4ab5"}
Oct 30 20:03:49.977309 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.977291    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:49.977472 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.977457    2579 scope.go:122] "RemoveContainer" containerID="1cec20fec7905162490d7ff0b1420979b6a2d6d2b80a98eb66677e509a427f58"
Oct 30 20:03:49.994452 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:49.994435    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:50.520366 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.520331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:50.619349 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.619310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:50.731030 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.730997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:50.731229 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:50.731109    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:50.757444 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.757415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:50.956056 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.955987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:50.982907 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.982887    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:03:50.983303 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.983266    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" event={"ID":"cdaa2197-530d-48e5-899c-5ccc58f4016a","Type":"ContainerStarted","Data":"3fb581cb1a50d1e348c3cc5a1a0c0871ece15d923f80ae794aa16346d9761b5c"}
Oct 30 20:03:50.983509 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.983489    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:50.983663 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.983645    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:50.998047 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:50.998027    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:03:51.020218 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:51.020151    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5" podStartSLOduration=8.776497643 podStartE2EDuration="23.020134134s" totalImagesPullingTime="14.243636491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:51.020099614 +0000 UTC m=+22.929974025" watchObservedRunningTime="2026-10-30 20:03:51.020134134 +0000 UTC m=+22.930008536"
Oct 30 20:03:51.156423 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:51.156387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:51.348998 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:51.348966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:51.729964 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:51.729940    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:51.729964 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:51.729959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:51.730366 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:51.730038    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:51.730366 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:51.730172    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:52.053632 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:52.053560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:52.153734 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:52.153707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:52.479127 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:52.479089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:52.730274 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:52.730210    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:52.730612 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:52.730343    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:53.730584 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:53.730557    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:53.730936 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:53.730659    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:53.730936 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:53.730686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:53.730936 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:53.730763    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:54.730987 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:54.730954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:54.731379 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:54.731078    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:55.730924 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:55.730893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:55.730924 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:55.730931    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:55.731481 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:55.731027    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:55.731481 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:55.731145    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:56.465475 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:56.465448    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:56.465656 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:56.465549    2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 20:03:56.465973 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:56.465955    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:03:56.730086 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:56.730022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:56.730235 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:56.730131    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:57.729930 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:57.729903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:57.730418 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:57.729902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:57.730418 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:57.730015    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:57.730418 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:57.730101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:03:58.731313 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:58.731267    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:03:58.731802 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:58.731365    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:03:59.730425 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:59.730390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:03:59.730425 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:03:59.730423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:03:59.730617 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:59.730503    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:03:59.730617 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:03:59.730567    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:04:00.729984 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:00.729945    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:00.730409 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:00.730061    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:04:01.415750 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:01.415714    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:01.639539 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:01.639503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:01.730051 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:01.730023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:01.730394 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:01.730060    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:01.730394 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:01.730154    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:04:01.730394 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:01.730198    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:04:01.770989 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:01.770965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:01.889935 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:01.889902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:02.189765 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.189731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:02.189894 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.189876    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:04:02.189951 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.189942    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret podName:3ce05b8e-185a-48d6-b18b-5858b2959885 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:34.189926014 +0000 UTC m=+66.099800414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret") pod "global-pull-secret-syncer-sgpdx" (UID: "3ce05b8e-185a-48d6-b18b-5858b2959885") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:04:02.290291 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.290257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:02.290454 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.290430    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:04:02.290526 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.290515    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs podName:4ea62f92-610f-4211-9153-474cd90cadaa nodeName:}" failed. No retries permitted until 2026-10-30 20:04:34.290493804 +0000 UTC m=+66.200368213 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs") pod "network-metrics-daemon-c9kfm" (UID: "4ea62f92-610f-4211-9153-474cd90cadaa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:04:02.391365 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.391084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:02.391497 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.391301    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:04:02.391497 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.391451    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:04:02.391497 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.391465    2579 projected.go:196] Error preparing data for projected volume kube-api-access-b78wd for pod openshift-network-diagnostics/network-check-target-zv26m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:02.391629 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.391530    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd podName:ed30ac6f-8649-4036-963c-25bef426479d nodeName:}" failed. No retries permitted until 2026-10-30 20:04:34.391512244 +0000 UTC m=+66.301386658 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b78wd" (UniqueName: "kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd") pod "network-check-target-zv26m" (UID: "ed30ac6f-8649-4036-963c-25bef426479d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:02.440946 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.440872    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zv26m"]
Oct 30 20:04:02.441103 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.440970    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:02.441103 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.441055    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:04:02.443961 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.443934    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sgpdx"]
Oct 30 20:04:02.444218 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.444195    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:02.444347 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.444327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:04:02.445487 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.445463    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c9kfm"]
Oct 30 20:04:02.445590 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.445553    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:02.445670 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:02.445650    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:04:02.531728 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.531690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:02.634911 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:02.634883    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:03.111979 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:03.111956    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzzv9_5efb380a-6530-4db0-88ec-2d59891344f1/dns-node-resolver/0.log"
Oct 30 20:04:03.729909 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:03.729880    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:03.729909 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:03.729912    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:03.730073 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:03.729990    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:04:03.730073 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:03.730041    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:03.730153 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:03.730138    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:04:03.730221 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:03.730202    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:04:04.541761 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:04.541733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:04.640930 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:04.640905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:04.712570 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:04.712547    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhts8_b5ddf8fa-c589-4ef9-8299-83ac15845b10/node-ca/0.log"
Oct 30 20:04:05.276883 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.276855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:05.410356 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.410327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:05.519777 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.519748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:05.633914 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.633844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:05.730238 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.730206    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:05.730388 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.730283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:05.730388 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:05.730372    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv26m" podUID="ed30ac6f-8649-4036-963c-25bef426479d"
Oct 30 20:04:05.730495 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.730380    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:05.730495 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:05.730463    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9kfm" podUID="4ea62f92-610f-4211-9153-474cd90cadaa"
Oct 30 20:04:05.730596 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:05.730526    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgpdx" podUID="3ce05b8e-185a-48d6-b18b-5858b2959885"
Oct 30 20:04:05.758694 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.758665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:05.861423 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.861394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:05.966094 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:05.966072    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:06.066533 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.066504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:06.868583 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.868562    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeReady"
Oct 30 20:04:06.868949 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.868655    2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 20:04:06.918868 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.918848    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ww6bn"]
Oct 30 20:04:06.937355 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.937334    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nnnfl"]
Oct 30 20:04:06.937485 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.937466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:06.944820 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.944797    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 20:04:06.944936 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.944912    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 20:04:06.944998 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.944958    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 20:04:06.945082 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.945063    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-nfbvj\""
Oct 30 20:04:06.957555 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.957536    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ww6bn"]
Oct 30 20:04:06.957631 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.957566    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nnnfl"]
Oct 30 20:04:06.957631 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.957594    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t7h8t"]
Oct 30 20:04:06.957691 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.957649    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:06.968847 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.968829    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:06.993339 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:06.993316    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t7h8t"]
Oct 30 20:04:07.022715 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.022662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81a971d1-9ef7-43cd-9f79-72ff95212079-cert\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.022715 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.022689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpmf\" (UniqueName: \"kubernetes.io/projected/81a971d1-9ef7-43cd-9f79-72ff95212079-kube-api-access-7kpmf\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.044597 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.044577    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 20:04:07.044597 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.044578    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 20:04:07.044738 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.044611    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 20:04:07.044876 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.044860    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 20:04:07.045217 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.045203    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 20:04:07.045717 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.045704    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 20:04:07.045980 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.045964    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c2z8r\""
Oct 30 20:04:07.046038 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.046000    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j5tm5\""
Oct 30 20:04:07.123321 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bprnz\" (UniqueName: \"kubernetes.io/projected/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-api-access-bprnz\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.123415 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd239462-09eb-4071-83f6-171abe1cd346-tmp-dir\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.123415 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd239462-09eb-4071-83f6-171abe1cd346-metrics-tls\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.123480 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab48927b-5485-4685-84d1-6141d4de8ceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.123480 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81a971d1-9ef7-43cd-9f79-72ff95212079-cert\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.123555 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpmf\" (UniqueName: \"kubernetes.io/projected/81a971d1-9ef7-43cd-9f79-72ff95212079-kube-api-access-7kpmf\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.123555 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd239462-09eb-4071-83f6-171abe1cd346-config-volume\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.123631 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab48927b-5485-4685-84d1-6141d4de8ceb-data-volume\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.123697 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.123752 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123715    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxq75\" (UniqueName: \"kubernetes.io/projected/dd239462-09eb-4071-83f6-171abe1cd346-kube-api-access-wxq75\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.123808 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.123764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab48927b-5485-4685-84d1-6141d4de8ceb-crio-socket\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.127491 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.127469    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81a971d1-9ef7-43cd-9f79-72ff95212079-cert\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.136241 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.136220    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpmf\" (UniqueName: \"kubernetes.io/projected/81a971d1-9ef7-43cd-9f79-72ff95212079-kube-api-access-7kpmf\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.224401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bprnz\" (UniqueName: \"kubernetes.io/projected/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-api-access-bprnz\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.224555 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd239462-09eb-4071-83f6-171abe1cd346-tmp-dir\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.224555 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd239462-09eb-4071-83f6-171abe1cd346-metrics-tls\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.224555 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab48927b-5485-4685-84d1-6141d4de8ceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.224703 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd239462-09eb-4071-83f6-171abe1cd346-config-volume\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.224703 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab48927b-5485-4685-84d1-6141d4de8ceb-data-volume\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.224703 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.224845 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxq75\" (UniqueName: \"kubernetes.io/projected/dd239462-09eb-4071-83f6-171abe1cd346-kube-api-access-wxq75\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.224845 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab48927b-5485-4685-84d1-6141d4de8ceb-crio-socket\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.224939 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224849    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd239462-09eb-4071-83f6-171abe1cd346-tmp-dir\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.224987 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.224953    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab48927b-5485-4685-84d1-6141d4de8ceb-crio-socket\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.225055 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.225037    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab48927b-5485-4685-84d1-6141d4de8ceb-data-volume\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.225306 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.225285    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd239462-09eb-4071-83f6-171abe1cd346-config-volume\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.225402 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.225306    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.226898 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.226878    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd239462-09eb-4071-83f6-171abe1cd346-metrics-tls\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.227011 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.226995    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab48927b-5485-4685-84d1-6141d4de8ceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.237031 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.237007    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bprnz\" (UniqueName: \"kubernetes.io/projected/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-api-access-bprnz\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.237128 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.237063    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxq75\" (UniqueName: \"kubernetes.io/projected/dd239462-09eb-4071-83f6-171abe1cd346-kube-api-access-wxq75\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.262155 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.262132    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:07.320970 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.320936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:07.365529 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.365498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:07.622659 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.622587    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ww6bn"]
Oct 30 20:04:07.625318 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.625287    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nnnfl"]
Oct 30 20:04:07.626843 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:07.626815    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a971d1_9ef7_43cd_9f79_72ff95212079.slice/crio-ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915 WatchSource:0}: Error finding container ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915: Status 404 returned error can't find the container with id ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915
Oct 30 20:04:07.627660 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:07.627639    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab48927b_5485_4685_84d1_6141d4de8ceb.slice/crio-7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d WatchSource:0}: Error finding container 7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d: Status 404 returned error can't find the container with id 7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d
Oct 30 20:04:07.659158 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.659109    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t7h8t"]
Oct 30 20:04:07.662418 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:07.662395    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd239462_09eb_4071_83f6_171abe1cd346.slice/crio-2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c WatchSource:0}: Error finding container 2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c: Status 404 returned error can't find the container with id 2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c
Oct 30 20:04:07.697106 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.697084    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-69d97cbb97-zgnfn"]
Oct 30 20:04:07.723101 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.723077    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69d97cbb97-zgnfn"]
Oct 30 20:04:07.723206 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.723168    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.728068 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728052    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fckx2\""
Oct 30 20:04:07.728703 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728682    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 20:04:07.728810 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728694    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 20:04:07.728810 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728767    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 20:04:07.728941 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728767    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 20:04:07.728941 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728792    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 20:04:07.728941 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728848    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 20:04:07.728941 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.728929    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 20:04:07.730213 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.730193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:07.730304 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.730218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:07.730364 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.730313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:07.733617 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.733602    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 20:04:07.733993 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.733980    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 20:04:07.734977 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.734958    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 20:04:07.735083 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.734974    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9mzsl\""
Oct 30 20:04:07.735083 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.735002    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pxq5r\""
Oct 30 20:04:07.735083 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.735012    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 20:04:07.735083 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.735053    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qkmrx\""
Oct 30 20:04:07.735307 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.735111    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.828666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.828817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.828861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.828892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.828964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.829012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.829065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.831410 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.829095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.930456 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.930456 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.930456 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.930456 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.930887    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.931336    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931401 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.931343    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931601 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.931527    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.931661 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.931641    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.933707 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.933690    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.933753 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.933712    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.943646 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.943627    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:07.985359 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:07.985335    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:08.016859 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.016835    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t7h8t" event={"ID":"dd239462-09eb-4071-83f6-171abe1cd346","Type":"ContainerStarted","Data":"2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c"}
Oct 30 20:04:08.018313 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.018293    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nnnfl" event={"ID":"ab48927b-5485-4685-84d1-6141d4de8ceb","Type":"ContainerStarted","Data":"9cfccbb8b029efd5561ceed60264490d4f71efb2647f0d60d5ce972cd1c50deb"}
Oct 30 20:04:08.018502 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.018339    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nnnfl" event={"ID":"ab48927b-5485-4685-84d1-6141d4de8ceb","Type":"ContainerStarted","Data":"7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d"}
Oct 30 20:04:08.019318 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.019298    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ww6bn" event={"ID":"81a971d1-9ef7-43cd-9f79-72ff95212079","Type":"ContainerStarted","Data":"ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915"}
Oct 30 20:04:08.045776 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.045755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:08.123526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.123490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:08.219638 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.219601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:08.346513 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.346479    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69d97cbb97-zgnfn"]
Oct 30 20:04:08.351184 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:08.351147    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a590085_c2de_41b6_aaff_a65de1a3a1c9.slice/crio-e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd WatchSource:0}: Error finding container e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd: Status 404 returned error can't find the container with id e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd
Oct 30 20:04:08.907601 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:08.907565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:09.008705 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.008675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:09.027252 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.026382    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69d97cbb97-zgnfn" event={"ID":"2a590085-c2de-41b6-aaff-a65de1a3a1c9","Type":"ContainerStarted","Data":"e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd"}
Oct 30 20:04:09.117959 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.117722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:09.255731 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.255697    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:09.361167 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.361105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:09.884373 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.884340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:09.994617 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:09.994463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.034088 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.034055    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t7h8t" event={"ID":"dd239462-09eb-4071-83f6-171abe1cd346","Type":"ContainerStarted","Data":"8884f115e864c5971fed0b4d78d4d890436c561ce25353667207426c89d78fc1"}
Oct 30 20:04:10.037953 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.036261    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nnnfl" event={"ID":"ab48927b-5485-4685-84d1-6141d4de8ceb","Type":"ContainerStarted","Data":"30e0fb560e000656bf65e30c5ed958a5d6bc19cf370b140faa1bab03e9e9e4c1"}
Oct 30 20:04:10.046756 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.046726    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"]
Oct 30 20:04:10.069766 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.068560    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d6vx6"]
Oct 30 20:04:10.069766 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.068866    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.082018 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.080676    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"]
Oct 30 20:04:10.082018 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.080734    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"]
Oct 30 20:04:10.082018 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.081030    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.082801 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.082677    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 20:04:10.083037 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.083014    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 20:04:10.083153 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.083109    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 20:04:10.083204 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.083180    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fg55m\""
Oct 30 20:04:10.084189 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.084148    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 20:04:10.084575 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.084556    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 20:04:10.092770 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.092749    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.098854 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.098472    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"]
Oct 30 20:04:10.114218 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.114180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.142402 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.142378    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 20:04:10.142511 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.142455    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 20:04:10.142673 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.142643    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 20:04:10.142775 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.142729    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kpzzd\""
Oct 30 20:04:10.142853 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.142801    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcdxj\""
Oct 30 20:04:10.142853 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.142832    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 20:04:10.173103 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.173008    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 20:04:10.174258 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.174236    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 20:04:10.236594 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.236556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.254364 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254330    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsn4c\" (UniqueName: \"kubernetes.io/projected/23c7a152-2247-4d92-9a1e-8f6a85b4954e-kube-api-access-vsn4c\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.254490 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftc9t\" (UniqueName: \"kubernetes.io/projected/26dcecb1-c6af-49e9-a840-b7197582b0cb-kube-api-access-ftc9t\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254490 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-root\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254490 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.254490 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-metrics-client-ca\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254637 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.254637 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.254716 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23c7a152-2247-4d92-9a1e-8f6a85b4954e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.254716 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.254716 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-tls\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254795 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w95dh\" (UniqueName: \"kubernetes.io/projected/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-api-access-w95dh\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.254795 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-wtmp\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254866 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254866 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-sys\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254866 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.254967 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.254967 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.255031 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.254979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-textfile\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.255031 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.255009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.342225 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.342190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.355458 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftc9t\" (UniqueName: \"kubernetes.io/projected/26dcecb1-c6af-49e9-a840-b7197582b0cb-kube-api-access-ftc9t\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355623 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-root\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355623 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.355623 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-metrics-client-ca\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355623 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.355801 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.355801 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23c7a152-2247-4d92-9a1e-8f6a85b4954e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.355801 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.355801 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-tls\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355985 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95dh\" (UniqueName: \"kubernetes.io/projected/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-api-access-w95dh\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.355985 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-wtmp\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355985 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355985 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-sys\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355985 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.355985 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.356283 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.355992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.356283 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.356034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-textfile\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.356283 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.356070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.356283 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.356110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsn4c\" (UniqueName: \"kubernetes.io/projected/23c7a152-2247-4d92-9a1e-8f6a85b4954e-kube-api-access-vsn4c\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.356283 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.356199    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-root\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.356848 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.356822    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-metrics-client-ca\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.356924 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.356900    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-sys\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.357077 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.357055    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23c7a152-2247-4d92-9a1e-8f6a85b4954e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.357612 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.357574    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.357612 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.357598    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.357755 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.357655    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-wtmp\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.358367 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.358037    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-textfile\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.358367 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.358260    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.358535 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.358513    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.360351 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.360328    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.361936 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.361917    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.362586 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.362524    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.364144 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.363860    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.364144 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.363981    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.364457 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.364434    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-tls\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.368788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.368762    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftc9t\" (UniqueName: \"kubernetes.io/projected/26dcecb1-c6af-49e9-a840-b7197582b0cb-kube-api-access-ftc9t\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.369139 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.369087    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsn4c\" (UniqueName: \"kubernetes.io/projected/23c7a152-2247-4d92-9a1e-8f6a85b4954e-kube-api-access-vsn4c\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.370332 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.370309    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95dh\" (UniqueName: \"kubernetes.io/projected/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-api-access-w95dh\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.431297 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.431250    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:10.444313 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.444280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.516690 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.516653    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:10.538358 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.538328    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:10.543467 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.543396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.715677 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.715598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.752450 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.752415    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"]
Oct 30 20:04:10.757908 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:10.757871    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c7a152_2247_4d92_9a1e_8f6a85b4954e.slice/crio-ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da WatchSource:0}: Error finding container ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da: Status 404 returned error can't find the container with id ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da
Oct 30 20:04:10.820223 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.820130    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"]
Oct 30 20:04:10.822154 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:10.821924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:10.824231 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:10.824199    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676bbe99_f475_40ee_9dfc_aa9cb8e8aa8e.slice/crio-5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1 WatchSource:0}: Error finding container 5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1: Status 404 returned error can't find the container with id 5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1
Oct 30 20:04:11.044922 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.044568    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw" event={"ID":"23c7a152-2247-4d92-9a1e-8f6a85b4954e","Type":"ContainerStarted","Data":"a9e390babecf879567abae69bc5c6786cc857a041cc8d01f9aac52df3d5f6934"}
Oct 30 20:04:11.044922 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.044638    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw" event={"ID":"23c7a152-2247-4d92-9a1e-8f6a85b4954e","Type":"ContainerStarted","Data":"012998ce867132a7aabc8dc5dbe6ea9e0b8365770dd0173d3eae7946733c6d01"}
Oct 30 20:04:11.044922 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.044667    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw" event={"ID":"23c7a152-2247-4d92-9a1e-8f6a85b4954e","Type":"ContainerStarted","Data":"ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da"}
Oct 30 20:04:11.048659 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.048628    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w" event={"ID":"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e","Type":"ContainerStarted","Data":"5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1"}
Oct 30 20:04:11.051109 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.051080    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t7h8t" event={"ID":"dd239462-09eb-4071-83f6-171abe1cd346","Type":"ContainerStarted","Data":"9e9c7ab9249bf9299512a282c2b6789fda0c3212c8cab9c2d6f9484fd3092f44"}
Oct 30 20:04:11.051430 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.051371    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:11.053056 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.053002    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6vx6" event={"ID":"26dcecb1-c6af-49e9-a840-b7197582b0cb","Type":"ContainerStarted","Data":"85cc4235496e2f6cc384db1909747d0e2e67bddb052a82e302f580f1a4763711"}
Oct 30 20:04:11.056646 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.056620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:11.071472 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.071420    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-t7h8t" podStartSLOduration=2.875390874 podStartE2EDuration="5.071402541s" totalImagesPullingTime="2.196011667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:11.070930894 +0000 UTC m=+42.980805307" watchObservedRunningTime="2026-10-30 20:04:11.071402541 +0000 UTC m=+42.981276953"
Oct 30 20:04:11.143940 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.142326    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:04:11.157685 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.157645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:11.161222 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.161197    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.164788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.164760    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:04:11.165835 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.165789    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 20:04:11.191816 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.189456    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9wn4n\""
Oct 30 20:04:11.191816 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.189566    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 20:04:11.258698 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.258620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:11.264472 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264633 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264633 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264633 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264626    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264788 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264968 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264968 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264968 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.264968 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.264878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.266291 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266270    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 20:04:11.266291 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266280    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 20:04:11.266291 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266273    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 20:04:11.266518 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266280    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 20:04:11.266518 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266337    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 20:04:11.266518 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266349    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 20:04:11.266518 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.266379    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 20:04:11.365960 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.365931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.365960 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.365965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366212 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366574 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366574 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366277    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366574 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.366300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.366574 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:11.366477    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle podName:51cefd7f-e086-40ff-b71b-c796ea9489e2 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:11.866453441 +0000 UTC m=+43.776327846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2") : configmap references non-existent config key: ca-bundle.crt
Oct 30 20:04:11.367035 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.367008    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.367108 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.367033    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.371183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.370871    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.371323 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.371190    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.371323 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.371210    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.371563 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.371518    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.372030 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.371719    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.372030 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.371995    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.372547 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.372526    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.372631 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.372545    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.373965 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.373936    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.383045 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.383025    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.871312 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.870995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.872372 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.872317    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:11.939333 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:11.939308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:12.060573 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.060540    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nnnfl" event={"ID":"ab48927b-5485-4685-84d1-6141d4de8ceb","Type":"ContainerStarted","Data":"ed74edbf8e4d272e03ef483e14349aa596afeff881aec7c153a16dd0eacf1dd7"}
Oct 30 20:04:12.064531 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.064498    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6vx6" event={"ID":"26dcecb1-c6af-49e9-a840-b7197582b0cb","Type":"ContainerStarted","Data":"c53df75307c0a5c1d353deb127f9768933620b3367725e29987d168073de35e7"}
Oct 30 20:04:12.071400 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.071370    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ww6bn" event={"ID":"81a971d1-9ef7-43cd-9f79-72ff95212079","Type":"ContainerStarted","Data":"c211ce1deda571c7ace1887107f6b316a08e3a954178d4b420ae2dc35a7a7644"}
Oct 30 20:04:12.089193 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.089108    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nnnfl" podStartSLOduration=2.413508873 podStartE2EDuration="6.089089948s" totalImagesPullingTime="3.675581075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:04:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:12.086329744 +0000 UTC m=+43.996204166" watchObservedRunningTime="2026-10-30 20:04:12.089089948 +0000 UTC m=+43.998964360"
Oct 30 20:04:12.107296 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.107237    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ww6bn" podStartSLOduration=2.160386956 podStartE2EDuration="6.107223606s" totalImagesPullingTime="3.94683665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:12.105797654 +0000 UTC m=+44.015672082" watchObservedRunningTime="2026-10-30 20:04:12.107223606 +0000 UTC m=+44.017098015"
Oct 30 20:04:12.140878 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.140853    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:12.246301 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.246261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:12.279888 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:12.279857    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:04:12.285512 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:12.285482    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cefd7f_e086_40ff_b71b_c796ea9489e2.slice/crio-55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df WatchSource:0}: Error finding container 55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df: Status 404 returned error can't find the container with id 55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df
Oct 30 20:04:13.061853 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.061679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:13.075965 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.075937    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df"}
Oct 30 20:04:13.078539 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.078513    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w" event={"ID":"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e","Type":"ContainerStarted","Data":"dd86efd55f269d3d4511626edfc98701d1f5a1f0aa825119bf8952fd707bea9c"}
Oct 30 20:04:13.078654 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.078608    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w" event={"ID":"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e","Type":"ContainerStarted","Data":"47bbdd958198999704023b9ca7374f151105ad0abd9271afd4bbd1050720799e"}
Oct 30 20:04:13.078654 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.078635    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w" event={"ID":"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e","Type":"ContainerStarted","Data":"beba6bc36e68548bed5f85c255489f8598fde7bea6bd6f6841694f74974a4f83"}
Oct 30 20:04:13.082202 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.082164    2579 generic.go:356] "Generic (PLEG): container finished" podID="26dcecb1-c6af-49e9-a840-b7197582b0cb" containerID="c53df75307c0a5c1d353deb127f9768933620b3367725e29987d168073de35e7" exitCode=0
Oct 30 20:04:13.082303 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.082230    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6vx6" event={"ID":"26dcecb1-c6af-49e9-a840-b7197582b0cb","Type":"ContainerDied","Data":"c53df75307c0a5c1d353deb127f9768933620b3367725e29987d168073de35e7"}
Oct 30 20:04:13.085971 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.085939    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw" event={"ID":"23c7a152-2247-4d92-9a1e-8f6a85b4954e","Type":"ContainerStarted","Data":"4df70b64fbc53c3a1d47c0fa865d2d397d6a0ce87d6b1aef318c1893ee27f3d0"}
Oct 30 20:04:13.101573 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.100741    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w" podStartSLOduration=1.279790159 podStartE2EDuration="3.100727333s" totalImagesPullingTime="1.820937174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:13.099872079 +0000 UTC m=+45.009746503" watchObservedRunningTime="2026-10-30 20:04:13.100727333 +0000 UTC m=+45.010601743"
Oct 30 20:04:13.122961 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.122907    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-69d97cbb97-zgnfn" podStartSLOduration=1.475195445 podStartE2EDuration="6.122889391s" totalImagesPullingTime="4.647693946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:13.121319149 +0000 UTC m=+45.031193560" watchObservedRunningTime="2026-10-30 20:04:13.122889391 +0000 UTC m=+45.032763804"
Oct 30 20:04:13.144355 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.143693    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw" podStartSLOduration=1.6704063200000001 podStartE2EDuration="3.143673196s" totalImagesPullingTime="1.473266876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:13.141402837 +0000 UTC m=+45.051277248" watchObservedRunningTime="2026-10-30 20:04:13.143673196 +0000 UTC m=+45.053547609"
Oct 30 20:04:13.177610 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.177569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:13.283534 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.283462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:13.486139 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.483847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:13.581189 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.581080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:13.742938 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.742899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:13.839214 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:13.839138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:14.090979 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.090898    2579 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74" exitCode=0
Oct 30 20:04:14.091436 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.090984    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74"}
Oct 30 20:04:14.092704 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.092679    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69d97cbb97-zgnfn" event={"ID":"2a590085-c2de-41b6-aaff-a65de1a3a1c9","Type":"ContainerStarted","Data":"b1b6f01507a5cf7e55d0c795a7d03f41052bdd438ad9e271ba5871dffb161a1b"}
Oct 30 20:04:14.094864 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.094843    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6vx6" event={"ID":"26dcecb1-c6af-49e9-a840-b7197582b0cb","Type":"ContainerStarted","Data":"7765bd58d641003a71217c2245feffb4c479d38381b895677a473102ec39dee6"}
Oct 30 20:04:14.094937 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.094883    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6vx6" event={"ID":"26dcecb1-c6af-49e9-a840-b7197582b0cb","Type":"ContainerStarted","Data":"820d49f030d86dc9bde887da410d719e06cbefe944a66660a7bbd573fa8148e2"}
Oct 30 20:04:14.148089 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.148033    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d6vx6" podStartSLOduration=2.685557036 podStartE2EDuration="4.148018841s" totalImagesPullingTime="1.365208062s" totalInitContainerRuntime="97.253743ms" isStatefulPod=true podCreationTimestamp="2026-10-30 20:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:14.146865594 +0000 UTC m=+46.056740003" watchObservedRunningTime="2026-10-30 20:04:14.148018841 +0000 UTC m=+46.057893250"
Oct 30 20:04:14.865837 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.865805    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"]
Oct 30 20:04:14.868060 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.868038    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:14.872946 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.872917    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 20:04:14.873566 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.873551    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8jpfj\""
Oct 30 20:04:14.879170 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.879147    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"]
Oct 30 20:04:14.888861 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.888812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:14.994531 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:14.994492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:15.015617 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.015579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8d6d749-b397-476f-83ca-774cc1aa5a0d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.015757 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.015675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9fnl\" (UniqueName: \"kubernetes.io/projected/a8d6d749-b397-476f-83ca-774cc1aa5a0d-kube-api-access-p9fnl\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.096190 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.096157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:15.117078 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.117006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8d6d749-b397-476f-83ca-774cc1aa5a0d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.117221 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.117170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9fnl\" (UniqueName: \"kubernetes.io/projected/a8d6d749-b397-476f-83ca-774cc1aa5a0d-kube-api-access-p9fnl\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.120712 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.120691    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8d6d749-b397-476f-83ca-774cc1aa5a0d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.130535 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.130514    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9fnl\" (UniqueName: \"kubernetes.io/projected/a8d6d749-b397-476f-83ca-774cc1aa5a0d-kube-api-access-p9fnl\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.181765 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.181736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:15.455674 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.455645    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"]
Oct 30 20:04:15.458590 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:15.458564    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d6d749_b397_476f_83ca_774cc1aa5a0d.slice/crio-b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e WatchSource:0}: Error finding container b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e: Status 404 returned error can't find the container with id b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e
Oct 30 20:04:15.913875 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.913847    2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69d97cbb97-zgnfn"]
Oct 30 20:04:15.952024 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.951997    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ddc794486-fvllw"]
Oct 30 20:04:15.953798 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.953784    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:15.966354 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:15.966334    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddc794486-fvllw"]
Oct 30 20:04:16.025253 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025224    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025355 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025355 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025451 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025499 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025460    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025499 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025610 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.025673 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.025621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.105674 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.105638    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc"}
Oct 30 20:04:16.106376 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.105725    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3"}
Oct 30 20:04:16.106376 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.105766    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9"}
Oct 30 20:04:16.106376 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.105803    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d"}
Oct 30 20:04:16.106376 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.105840    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262"}
Oct 30 20:04:16.106939 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.106916    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l" event={"ID":"a8d6d749-b397-476f-83ca-774cc1aa5a0d","Type":"ContainerStarted","Data":"b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e"}
Oct 30 20:04:16.126335 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126420 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126420 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126420 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126420 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126420 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126564 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.126564 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.126476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.127454 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.127426    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.127643 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.127619    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.127705 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.127644    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.127705 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.127630    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.129103 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.127973    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.129553 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.129522    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.131103 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.131079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.141235 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.141205    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.263183 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.263137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:16.416093 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.416055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:16.546159 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:16.545908    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddc794486-fvllw"]
Oct 30 20:04:16.549105 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:16.549041    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7a756f_0197_4cfb_bb53_dfde71285608.slice/crio-f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b WatchSource:0}: Error finding container f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b: Status 404 returned error can't find the container with id f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b
Oct 30 20:04:17.112358 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.112274    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddc794486-fvllw" event={"ID":"ac7a756f-0197-4cfb-bb53-dfde71285608","Type":"ContainerStarted","Data":"cbd97c9332b35f673d94331356ca8862c73fe8f6f73c1502d47b3ca1723dff39"}
Oct 30 20:04:17.112358 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.112329    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddc794486-fvllw" event={"ID":"ac7a756f-0197-4cfb-bb53-dfde71285608","Type":"ContainerStarted","Data":"f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b"}
Oct 30 20:04:17.114307 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.114279    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l" event={"ID":"a8d6d749-b397-476f-83ca-774cc1aa5a0d","Type":"ContainerStarted","Data":"2a02436728df715d697d202ddb3e44a22e08543e7f6335e70d6f7d972ad000e9"}
Oct 30 20:04:17.114526 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.114466    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:17.118464 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.118438    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerStarted","Data":"8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610"}
Oct 30 20:04:17.119806 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.119787    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:17.136466 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.136425    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7ddc794486-fvllw" podStartSLOduration=2.136412688 podStartE2EDuration="2.136412688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:17.134536241 +0000 UTC m=+49.044410651" watchObservedRunningTime="2026-10-30 20:04:17.136412688 +0000 UTC m=+49.046287097"
Oct 30 20:04:17.174481 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.174396    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.667228234 podStartE2EDuration="6.174377232s" totalImagesPullingTime="3.493148936s" totalInitContainerRuntime="14.000062ms" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:17.17232205 +0000 UTC m=+49.082196459" watchObservedRunningTime="2026-10-30 20:04:17.174377232 +0000 UTC m=+49.084251646"
Oct 30 20:04:17.196047 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:17.195998    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l" podStartSLOduration=1.821804015 podStartE2EDuration="3.195982926s" totalImagesPullingTime="1.374178911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:17.195035138 +0000 UTC m=+49.104909550" watchObservedRunningTime="2026-10-30 20:04:17.195982926 +0000 UTC m=+49.105857336"
Oct 30 20:04:18.046393 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:18.046364    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:19.626250 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:19.626196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:19.730714 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:19.730666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:21.073884 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:21.073851    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:22.114705 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:22.114673    2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddc794486-fvllw"]
Oct 30 20:04:22.627635 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:22.627598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:22.731104 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:22.731067    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:22.997468 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:22.997436    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:23.174854 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:23.174823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:23.287982 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:23.287907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:23.543448 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:23.543365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:23.660405 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:23.660369    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:23.777970 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:23.777943    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:23.891489 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:23.891422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:24.013855 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:24.013825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:24.166237 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:24.166151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:24.292382 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:24.292346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:24.396482 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:24.396456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:24.740327 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:24.740296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:24.847956 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:24.847926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:25.559563 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:25.559531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:25.659963 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:25.659931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:26.264193 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:26.264163    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:32.632935 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:32.632899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:32.766112 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:32.766084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:34.193862 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.193836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:34.283108 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.283078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:34.285959 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.285942    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 20:04:34.295302 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.295271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:34.296418 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.296396    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:34.383860 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.383837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:34.386591 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.386571    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 20:04:34.396849 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.396828    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:34.468304 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.468278    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qkmrx\""
Oct 30 20:04:34.476332 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.476313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:34.484660 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.484639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:34.487404 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.487385    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 20:04:34.490373 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.490313    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pxq5r\""
Oct 30 20:04:34.497892 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.497875    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 20:04:34.497984 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.497894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:34.508518 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.508497    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:34.533254 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.533018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:34.641028 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.641000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:34.754206 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.754139    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sgpdx"]
Oct 30 20:04:34.756769 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:34.756744    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce05b8e_185a_48d6_b18b_5858b2959885.slice/crio-23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe WatchSource:0}: Error finding container 23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe: Status 404 returned error can't find the container with id 23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe
Oct 30 20:04:34.758453 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.758434    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9mzsl\""
Oct 30 20:04:34.766739 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.766721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:34.784141 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:34.784105    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c9kfm"]
Oct 30 20:04:34.785906 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:34.785878    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea62f92_610f_4211_9153_474cd90cadaa.slice/crio-aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde WatchSource:0}: Error finding container aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde: Status 404 returned error can't find the container with id aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde
Oct 30 20:04:35.038448 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:35.038345    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zv26m"]
Oct 30 20:04:35.040913 ip-10-0-129-14 kubenswrapper[2579]: W1030 20:04:35.040882    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded30ac6f_8649_4036_963c_25bef426479d.slice/crio-14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9 WatchSource:0}: Error finding container 14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9: Status 404 returned error can't find the container with id 14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9
Oct 30 20:04:35.179620 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:35.179588    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c9kfm" event={"ID":"4ea62f92-610f-4211-9153-474cd90cadaa","Type":"ContainerStarted","Data":"aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde"}
Oct 30 20:04:35.180734 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:35.180712    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zv26m" event={"ID":"ed30ac6f-8649-4036-963c-25bef426479d","Type":"ContainerStarted","Data":"14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9"}
Oct 30 20:04:35.181795 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:35.181775    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sgpdx" event={"ID":"3ce05b8e-185a-48d6-b18b-5858b2959885","Type":"ContainerStarted","Data":"23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe"}
Oct 30 20:04:36.189720 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:36.189631    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c9kfm" event={"ID":"4ea62f92-610f-4211-9153-474cd90cadaa","Type":"ContainerStarted","Data":"ebfa286c992b57f2b73ff538b6338aa554c46b5979d1fd533ec414307cbd2df1"}
Oct 30 20:04:36.190209 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:36.189727    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c9kfm" event={"ID":"4ea62f92-610f-4211-9153-474cd90cadaa","Type":"ContainerStarted","Data":"1545193bf76f60a093dd3e9383cabda5ea53b8a277177900b165ffec4cb73957"}
Oct 30 20:04:36.209365 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:36.209300    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c9kfm" podStartSLOduration=67.15940254 podStartE2EDuration="1m8.209284898s" totalImagesPullingTime="1.049882358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:36.208589694 +0000 UTC m=+68.118464108" watchObservedRunningTime="2026-10-30 20:04:36.209284898 +0000 UTC m=+68.119159309"
Oct 30 20:04:36.341974 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:36.341940    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:36.442373 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:36.442226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:37.353343 ip-10-0-129-14 kubenswrapper[2579]: E1030 20:04:37.353302    2579 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:04:39.204827 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:39.204790    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zv26m" event={"ID":"ed30ac6f-8649-4036-963c-25bef426479d","Type":"ContainerStarted","Data":"3f85d16f59750c39282685a71bb5603523056536164d4e19b13646725531976f"}
Oct 30 20:04:39.205337 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:39.204939    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:39.230809 ip-10-0-129-14 kubenswrapper[2579]: I1030 20:04:39.230749    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zv26m" podStartSLOduration=67.885575359 podStartE2EDuration="1m11.230734292s" totalImagesPullingTime="3.345158933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:39.229990757 +0000 UTC m=+71.139865165" watchObservedRunningTime="2026-10-30 20:04:39.230734292 +0000 UTC m=+71.140608703"
Oct 30 20:04:39.742002 ip-10-0-129-14 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 20:04:39.760484 ip-10-0-129-14 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 20:04:39.760659 ip-10-0-129-14 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 20:04:39.760836 ip-10-0-129-14 systemd[1]: kubelet.service: Consumed 4.270s CPU time.
Oct 30 20:04:39.769782 ip-10-0-129-14 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 20:04:39.827899 ip-10-0-129-14 kubenswrapper[6389]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:04:39.827899 ip-10-0-129-14 kubenswrapper[6389]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 20:04:39.827899 ip-10-0-129-14 kubenswrapper[6389]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:04:39.827899 ip-10-0-129-14 kubenswrapper[6389]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:04:39.831090 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831008    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.831090 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831088    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.831090 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831093    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.831090 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831096    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831099    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831102    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831105    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831108    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831111    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831126    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831130    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831133    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831136    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831139    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831142    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831144    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831148    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831150    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831153    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831156    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831160    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831165    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.831292 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831168    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831171    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831174    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831176    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831184    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831187    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831189    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831192    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831195    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831205    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831208    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831211    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831214    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831217    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831219    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831222    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831225    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831227    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831230    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831233    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.831775 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831236    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831238    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831241    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831244    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831246    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831249    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831252    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831255    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831257    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831260    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831263    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831266    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831269    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831271    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831274    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831277    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831281    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831284    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831293    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831297    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.832307 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831300    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831303    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831306    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831309    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831311    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831314    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831317    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831321    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831323    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831326    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831329    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831331    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831334    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831337    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831340    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831342    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831345    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831348    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831350    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.832788 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831353    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831356    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831360    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831363    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831366    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831369    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831371    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831374    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831377    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831381    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831386    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831389    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831393    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831396    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831399    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831401    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831404    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831407    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831410    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831412    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.833305 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831416    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831419    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831421    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831424    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.831427    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832194    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832202    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832204    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832207    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832210    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832213    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832216    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832219    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832222    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832225    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832229    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832231    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832234    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832237    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.833803 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832240    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832242    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832246    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832248    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832251    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832253    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832256    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832259    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832263    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832267    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832270    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832273    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832276    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832278    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832281    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832284    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832286    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832289    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832292    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.834274 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832294    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832297    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832299    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832302    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832305    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832308    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832311    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832315    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832317    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832320    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832324    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832327    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832330    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832333    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832335    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832338    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832341    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832344    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832346    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832349    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.834798 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832352    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832354    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832357    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832359    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832362    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832365    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832367    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832370    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832373    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832375    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832378    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832380    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832383    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832386    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832389    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832391    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832394    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832397    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832400    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.835388 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832402    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832405    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832408    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832411    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832414    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832416    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832419    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832422    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832424    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832427    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832429    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832432    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832434    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832437    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832440    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832444    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832448    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832451    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832454    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.835970 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832456    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832460    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832463    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832465    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832468    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832470    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832473    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832476    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832479    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832482    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832484    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832487    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832490    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832492    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832496    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832984    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832990    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832994    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832996    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.832999    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.836794 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833002    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833005    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833008    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833011    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833014    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833017    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833020    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833023    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833027    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833030    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833033    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833035    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833038    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833041    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833044    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833047    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833050    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833053    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833056    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.837564 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833059    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833061    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833064    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833066    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833069    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833072    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833074    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833077    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833080    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833082    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833085    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833088    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833091    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833093    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833096    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833099    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833102    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833106    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833108    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.838169 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833111    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833129    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833136    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833140    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833145    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833148    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833151    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833154    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833156    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833159    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833162    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833165    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833167    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833170    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833173    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833175    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833178    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833180    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833183    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.838638 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833186    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833188    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833191    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833193    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833196    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833198    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833202    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833205    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833208    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833210    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833213    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833216    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833218    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833221    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833224    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833229    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833233    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833236    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833239    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833242    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.839106 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833244    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833247    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833249    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833252    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833255    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833258    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833260    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833263    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833266    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833268    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833271    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833274    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833276    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833279    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833281    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833284    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833287    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833289    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833292    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833296    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.839737 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833299    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833301    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833304    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.833307    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833398    6389 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833406    6389 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833411    6389 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833417    6389 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833422    6389 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833425    6389 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833432    6389 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833436    6389 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833439    6389 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833442    6389 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833446    6389 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833450    6389 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833453    6389 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833456    6389 flags.go:64] FLAG: --cgroup-root=""
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833461    6389 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833464    6389 flags.go:64] FLAG: --client-ca-file=""
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833467    6389 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833470    6389 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833478    6389 flags.go:64] FLAG: --cluster-domain=""
Oct 30 20:04:39.840270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833481    6389 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833484    6389 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833487    6389 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833491    6389 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833495    6389 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833498    6389 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833501    6389 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833504    6389 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833508    6389 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833511    6389 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833514    6389 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833517    6389 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833521    6389 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833525    6389 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833528    6389 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833532    6389 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833536    6389 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833539    6389 flags.go:64] FLAG: --enable-server="true"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833542    6389 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833549    6389 flags.go:64] FLAG: --event-burst="100"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833553    6389 flags.go:64] FLAG: --event-qps="50"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833555    6389 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833559    6389 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833562    6389 flags.go:64] FLAG: --eviction-hard=""
Oct 30 20:04:39.840842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833566    6389 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833569    6389 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833572    6389 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833576    6389 flags.go:64] FLAG: --eviction-soft=""
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833579    6389 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833582    6389 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833585    6389 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833588    6389 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833591    6389 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833594    6389 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833597    6389 flags.go:64] FLAG: --feature-gates=""
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833601    6389 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833604    6389 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833608    6389 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833611    6389 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833615    6389 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833618    6389 flags.go:64] FLAG: --help="false"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833621    6389 flags.go:64] FLAG: --hostname-override="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833625    6389 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833628    6389 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833631    6389 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833634    6389 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833637    6389 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 20:04:39.841603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833640    6389 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833645    6389 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833648    6389 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833651    6389 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833654    6389 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833657    6389 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833660    6389 flags.go:64] FLAG: --kube-reserved=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833663    6389 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833666    6389 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833669    6389 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833672    6389 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833675    6389 flags.go:64] FLAG: --lock-file=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833680    6389 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833683    6389 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833686    6389 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833691    6389 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833694    6389 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833698    6389 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833701    6389 flags.go:64] FLAG: --logging-format="text"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833704    6389 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833707    6389 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833710    6389 flags.go:64] FLAG: --manifest-url=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833713    6389 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833717    6389 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833720    6389 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 20:04:39.842283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833724    6389 flags.go:64] FLAG: --max-pods="110"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833728    6389 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833731    6389 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833734    6389 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833737    6389 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833740    6389 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833743    6389 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833746    6389 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833754    6389 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833758    6389 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833761    6389 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833764    6389 flags.go:64] FLAG: --pod-cidr=""
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833767    6389 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833770    6389 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833773    6389 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833776    6389 flags.go:64] FLAG: --port="10250"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833779    6389 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833782    6389 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0dade436e37cb95e7"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833785    6389 flags.go:64] FLAG: --qos-reserved=""
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833790    6389 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833793    6389 flags.go:64] FLAG: --register-node="true"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833796    6389 flags.go:64] FLAG: --register-with-taints=""
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833800    6389 flags.go:64] FLAG: --registry-burst="10"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833803    6389 flags.go:64] FLAG: --registry-qps="5"
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833806    6389 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833809    6389 flags.go:64] FLAG: --reserved-memory=""
Oct 30 20:04:39.843224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833813    6389 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833816    6389 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833819    6389 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833822    6389 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833826    6389 flags.go:64] FLAG: --runonce="false"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833829    6389 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833832    6389 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833835    6389 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833837    6389 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833841    6389 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833845    6389 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833848    6389 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833851    6389 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833854    6389 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833859    6389 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833862    6389 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833865    6389 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833868    6389 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833871    6389 flags.go:64] FLAG: --system-cgroups=""
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833874    6389 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833879    6389 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833882    6389 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833885    6389 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833889    6389 flags.go:64] FLAG: --tls-min-version=""
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833892    6389 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 20:04:39.844272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833896    6389 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833899    6389 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833902    6389 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833905    6389 flags.go:64] FLAG: --v="2"
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833910    6389 flags.go:64] FLAG: --version="false"
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833914    6389 flags.go:64] FLAG: --vmodule=""
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833918    6389 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.833921    6389 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834066    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834070    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834073    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834077    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834080    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834083    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834085    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834088    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834091    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834094    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834097    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834100    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834102    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.845203 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834106    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834109    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834112    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834129    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834133    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834138    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834142    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834146    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834149    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834152    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834155    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834159    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834162    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834165    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834168    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834171    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834173    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834176    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834179    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834182    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.846087 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834184    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834187    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834190    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834193    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834195    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834198    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834201    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834204    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834206    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834209    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834213    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834216    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834220    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834223    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834226    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834229    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834231    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834235    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834238    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.846956 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834240    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834244    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834248    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834251    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834256    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834259    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834262    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834265    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834269    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834271    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834274    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834277    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834280    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834283    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834286    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834289    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834292    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834294    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834297    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.847877 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834300    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834303    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834305    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834308    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834311    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834319    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834324    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834327    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834329    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834332    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834335    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834338    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834340    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834343    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834345    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834348    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834351    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834355    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834358    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834361    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.848594 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834363    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834366    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834369    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834371    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834374    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834378    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834382    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834385    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834388    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834391    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834393    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834396    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834399    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834402    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834404    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.849276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.834430    6389 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834469    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834473    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834482    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834487    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834490    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834493    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834495    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834498    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834501    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834504    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834506    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834509    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834512    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834515    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834518    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834521    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834524    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834527    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.849724 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834529    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834532    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834535    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834537    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834540    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834543    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834545    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834548    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834552    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834556    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834560    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834564    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834568    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834572    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834577    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834581    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834583    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834599    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834603    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.850610 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834606    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834609    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834612    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834615    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834618    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834620    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834623    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834626    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834630    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834633    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834636    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834638    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834641    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834644    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834648    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834650    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834653    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834656    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834658    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834661    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.851419 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834664    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834666    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834670    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834674    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834677    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834680    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834683    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834685    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834688    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834691    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834695    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834707    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834710    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834713    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834716    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834718    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834721    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834724    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834726    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.852079 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834729    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834734    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834736    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834739    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834742    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834744    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834747    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834750    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834752    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834755    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834757    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834760    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834763    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834765    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834768    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834771    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834773    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834776    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834779    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834781    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.852757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834784    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834786    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834789    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834793    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834796    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834804    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834810    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834813    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834817    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.834820    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.839535    6389 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.839551    6389 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.839570    6389 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.839575    6389 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839653    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839658    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839663    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839668    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839673    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.853405 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839678    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839681    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839684    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839686    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839689    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839692    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839695    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839697    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839700    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839703    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839706    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839709    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839717    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839720    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839724    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839727    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839729    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839732    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839741    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.853889 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839744    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839747    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839750    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839753    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839756    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839759    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839762    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839765    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839767    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839770    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839773    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839776    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839778    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839781    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839784    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839786    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839791    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839795    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839798    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.854376 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839801    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839804    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839807    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839811    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839814    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839817    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839820    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839823    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839826    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839829    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839831    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839834    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839837    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839846    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839849    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839851    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839854    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839857    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839860    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839862    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.854846 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839865    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839868    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839870    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839873    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839876    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839879    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839882    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839884    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839887    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839890    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839892    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839895    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839898    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839900    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839903    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839906    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839909    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839912    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839914    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.855369 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839917    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839920    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839923    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839926    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839929    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839932    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839935    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839944    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839947    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839950    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839953    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839955    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839958    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839962    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839966    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839968    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839971    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839974    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839977    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.856016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839979    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839982    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839985    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839988    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.839991    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.840018    6389 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840217    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840223    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840226    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840229    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840232    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840235    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840238    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840240    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840243    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840246    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.856570 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840248    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840251    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840253    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840256    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840260    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840268    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840271    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840274    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840277    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840280    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840283    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840286    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840289    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840291    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840294    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840296    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840299    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840302    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840304    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.856968 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840307    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840310    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840312    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840315    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840318    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840320    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840323    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840325    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840328    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840331    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840334    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840336    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840339    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840342    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840344    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840347    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840350    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840352    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840355    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840363    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.857452 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840366    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840369    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840371    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840374    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840377    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840379    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840382    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840385    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840387    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840390    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840393    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840395    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840398    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840402    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840406    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840409    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840412    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840414    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840418    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.858016 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840422    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840425    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840429    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840431    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840434    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840437    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840440    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840442    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840445    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840447    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840450    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840452    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840455    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840464    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840466    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840469    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840472    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840475    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840477    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840480    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.858823 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840482    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840485    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840488    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840490    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840493    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840496    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840498    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840501    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840503    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840506    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840509    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840511    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840514    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840517    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840519    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840522    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840525    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.859542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840527    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.840551    6389 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840618    6389 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840627    6389 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840632    6389 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840636    6389 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840639    6389 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840642    6389 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840645    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840653    6389 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840656    6389 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840659    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840662    6389 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840665    6389 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840667    6389 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840670    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:04:39.860164 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840673    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840676    6389 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840678    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840681    6389 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840684    6389 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840687    6389 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840689    6389 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840692    6389 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840695    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840700    6389 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840705    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840710    6389 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840713    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840716    6389 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840725    6389 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840728    6389 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840731    6389 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840733    6389 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840736    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840739    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:04:39.860723 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840741    6389 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840744    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840747    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840749    6389 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840752    6389 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840754    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840762    6389 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840766    6389 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840768    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840771    6389 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840775    6389 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840780    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840785    6389 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840789    6389 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840792    6389 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840794    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840797    6389 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840799    6389 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840802    6389 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840805    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:04:39.861421 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840807    6389 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840810    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840813    6389 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840815    6389 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840818    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840820    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840823    6389 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840826    6389 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840829    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840831    6389 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840834    6389 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840837    6389 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840839    6389 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840842    6389 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840845    6389 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840847    6389 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840850    6389 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840854    6389 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840859    6389 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840871    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:04:39.861924 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840875    6389 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840877    6389 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840880    6389 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840883    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840886    6389 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840888    6389 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840891    6389 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840893    6389 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840896    6389 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840899    6389 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840902    6389 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840904    6389 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840907    6389 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840910    6389 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840912    6389 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840915    6389 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840917    6389 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840923    6389 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840926    6389 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:04:39.862466 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840928    6389 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840932    6389 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840936    6389 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840941    6389 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840946    6389 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840949    6389 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840952    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840955    6389 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840959    6389 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840963    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840966    6389 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840975    6389 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:04:39.840978    6389 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.841040    6389 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.842162    6389 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.842220    6389 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.842421    6389 server.go:1014] "Starting client certificate rotation"
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.842563    6389 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:04:39.862972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.843172    6389 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.843310    6389 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:58:28 +0000 UTC" deadline="2028-06-07 00:43:17.416637958 +0000 UTC"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.843359    6389 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14044h38m37.573282634s"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.845224    6389 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.848998    6389 log.go:25] "Validated CRI v1 runtime API"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.851468    6389 log.go:25] "Validated CRI v1 image API"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.852375    6389 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 20:04:39.863567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.854355    6389 fs.go:135] Filesystem UUIDs: map[02667abc-60a6-46d9-b3be-b2d61d786408:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e2e521f0-2ae4-42d4-a14d-b482c126cc6b:/dev/nvme0n1p4]
Oct 30 20:04:39.863793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.854370    6389 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89/userdata/shm major:0 minor:97 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9/userdata/shm major:0 minor:405 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe/userdata/shm major:0 minor:398 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1/userdata/shm major:0 minor:257 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ce83785c43575b988dfc59ee502446025dde26b6a67fe37d3a4e1784bc14e53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ce83785c43575b988dfc59ee502446025dde26b6a67fe37d3a4e1784bc14e53/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/85cc4235496e2f6cc384db1909747d0e2e67bddb052a82e302f580f1a4763711/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/85cc4235496e2f6cc384db1909747d0e2e67bddb052a82e302f580f1a4763711/userdata/shm major:0 minor:259 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a40bd710d363b612693bc3c34aa642eaae991f884783c59ef78f254aecbddd48/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a40bd710d363b612693bc3c34aa642eaae991f884783c59ef78f254aecbddd48/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde/userdata/shm major:0 minor:400 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da/userdata/shm major:0 minor:255 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b/userdata/shm major:0 minor:375 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~projected/kube-api-access-vsn4c:{mountpoint:/var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~projected/kube-api-access-vsn4c major:0 minor:253 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:250 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~projected/kube-api-access-ftc9t:{mountpoint:/var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~projected/kube-api-access-ftc9t major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~projected/kube-api-access-5fzc7:{mountpoint:/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~projected/kube-api-access-5fzc7 major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ce05b8e-185a-48d6-b18b-5858b2959885/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/3ce05b8e-185a-48d6-b18b-5858b2959885/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:392 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41/volumes/kubernetes.io~projected/kube-api-access-t8gkt:{mountpoint:/var/lib/kubelet/pods/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41/volumes/kubernetes.io~projected/kube-api-access-t8gkt major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ea62f92-610f-4211-9153-474cd90cadaa/volumes/kubernetes.io~projected/kube-api-access-q59hq:{mountpoint:/var/lib/kubelet/pods/4ea62f92-610f-4211-9153-474cd90cadaa/volumes/kubernetes.io~projected/kube-api-access-q59hq major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ea62f92-610f-4211-9153-474cd90cadaa/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/4ea62f92-610f-4211-9153-474cd90cadaa/volumes/kubernetes.io~secret/metrics-certs major:0 minor:397 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/alertmanager/9 major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/config-reloader/6 major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~empty-dir/config-out major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~projected/kube-api-access-xtshj:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~projected/kube-api-access-xtshj major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~projected/tls-assets major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/config-volume major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/web-config major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5efb380a-6530-4db0-88ec-2d59891344f1/volumes/kubernetes.io~projected/kube-api-access-dk2l2:{mountpoint:/var/lib/kubelet/pods/5efb380a-6530-4db0-88ec-2d59891344f1/volumes/kubernetes.io~projected/kube-api-access-dk2l2 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~projected/kube-api-access-w95dh:{mountpoint:/var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~projected/kube-api-access-w95dh major:0 minor:254 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:249 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:247 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ba6f137-f70b-41a4-9e2b-871bb10b4f07/volumes/kubernetes.io~projected/kube-api-access-7tr6l:{mountpoint:/var/lib/kubelet/pods/6ba6f137-f70b-41a4-9e2b-871bb10b4f07/volumes/kubernetes.io~projected/kube-api-access-7tr6l major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81a971d1-9ef7-43cd-9f79-72ff95212079/volumes/kubernetes.io~projected/kube-api-access-7kpmf:{mountpoint:/var/lib/kubelet/pods/81a971d1-9ef7-43cd-9f79-72ff95212079/volumes/kubernetes.io~projected/kube-api-access-7kpmf major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81a971d1-9ef7-43cd-9f79-72ff95212079/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/81a971d1-9ef7-43cd-9f79-72ff95212079/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8bdc9936-09aa-4583-95e7-499428c2b9b0/volumes/kubernetes.io~projected/kube-api-access-sdc9g:{mountpoint:/var/lib/kubelet/pods/8bdc9936-09aa-4583-95e7-499428c2b9b0/volumes/kubernetes.io~projected/kube-api-access-sdc9g major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8d6d749-b397-476f-83ca-774cc1aa5a0d/volumes/kubernetes.io~projected/kube-api-access-p9fnl:{mountpoint:/var/lib/kubelet/pods/a8d6d749-b397-476f-83ca-774cc1aa5a0d/volumes/kubernetes.io~projected/kube-api-access-p9fnl major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8d6d749-b397-476f-83ca-774cc1aa5a0d/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/a8d6d749-b397-476f-83ca-774cc1aa5a0d/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab48927b-5485-4685-84d1-6141d4de8ceb/volumes/kubernetes.io~projected/kube-api-access-bprnz:{mountpoint:/var/lib/kubelet/pods/ab48927b-5485-4685-84d1-6141d4de8ceb/volumes/kubernetes.io~projected/kube-api-access-bprnz major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab48927b-5485-4685-84d1-6141d4de8ceb/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/ab48927b-5485-4685-84d1-6141d4de8ceb/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~projected/kube-api-access-z2fzg:{mountpoint:/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~projected/kube-api-access-z2fzg major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b5ddf8fa-c589-4ef9-8299-83ac15845b10/volumes/kubernetes.io~projected/kube-api-access-pc8vv:{mountpoint:/var/lib/kubelet/pods/b5ddf8fa-c589-4ef9-8299-83ac15845b10/volumes/kubernetes.io~projected/kube-api-access-pc8vv major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volumes/kubernetes.io~projected/kube-api-access-zzlpm:{mountpoint:/var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volumes/kubernetes.io~projected/kube-api-access-zzlpm major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dd239462-09eb-4071-83f6-171abe1cd346/volumes/kubernetes.io~projected/kube-api-access-wxq75:{mountpoint:/var/lib/kubelet/pods/dd239462-09eb-4071-83f6-171abe1cd346/volumes/kubernetes.io~projected/kube-api-access-wxq75 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dd239462-09eb-4071-83f6-171abe1cd346/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/dd239462-09eb-4071-83f6-171abe1cd346/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de259d4b-08fb-48bb-9aad-0a381205090f/volumes/kubernetes.io~projected/kube-api-access-s7wjj:{mountpoint:/var/lib/kubelet/pods/de259d4b-08fb-48bb-9aad-0a381205090f/volumes/kubernetes.io~projected/kube-api-access-s7wjj major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eab05532-1d94-4218-bef2-a7f43bfd25e8/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/eab05532-1d94-4218-bef2-a7f43bfd25e8/volumes/kubernetes.io~secret/agent-certs major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed30ac6f-8649-4036-963c-25bef426479d/volumes/kubernetes.io~projected/kube-api-access-b78wd:{mountpoint:/var/lib/kubelet/pods/ed30ac6f-8649-4036-963c-25bef426479d/volumes/kubernetes.io~projected/kube-api-access-b78wd major:0 minor:402 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~empty-dir/tmp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~projected/kube-api-access-tgrsp:{mountpoint:/var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~projected/kube-api-access-tgrsp major:0 minor:62 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/5691c657e2f174c12db8ffead21094e6ccb46013f7673eb22c9b7c45be691daf/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/54cb0f047dbb8f26373b21400f6b8121d90840b792d789e3d3d02c864010dd50/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/4b0f12f2e093cd5a143591052cb4b3bfe7322cffdaee2596d3a277ded1b7bea1/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/daf6cbb9be9ba9a57275de80dbb48ca371a46b093b7c4dce4d59969343a15759/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/3993ec43fd70351750a8872e522df67c5ab10d3490dc6ab4b5fc20ed3542a8b3/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/c0000b2b55a5e7803432f7d2f71241b85e0fbc7a47a14474e374a9627534dbb8/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/559cd28bc07bf59119ac02784b09c9bca1d9cf30572908a37d50bf65ff3445d4/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/217328e8075991094cf860a151c7a8be65eb39439dd8c7cc79aeb7ef4ceee4b2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/2656fb91192bca0b187fdae99830137e12511bfffc2ff369d26691243761f78e/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/4967e22eecd0a2f24beddc3203bb4640ce5e2fe2e1c043df162677f55af9630b/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/b1ae6d77a6600fd37dc60987fb2dd6a8e7ebf6954d6e1c32e96c697af8620fac/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/ba83d6863c77e19d3faa393d1b29302319031f586304964ca686e0f5d89828f3/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-153:{mountpoint:/var/lib/containers/storage/overlay/b916c01c11a1326b10347be8ca0d8ce28ffd7e544f3ce4720c78da637949325f/merged major:0 minor:153 fsType:overlay blockSize:0} overlay_0-155:{mountpoint:/var/lib/containers/storage/overlay/6924ac03f5add69bbd0a07e0ea141fe49bdbb9a24737d1a228b8c9ee470813f9/merged major:0 minor:155 fsType:overlay blockSize:0} overlay_0-156:{mountpoint:/var/lib/containers/storage/overlay/c5ea567bb09d90cb80f1ed42d6adb28617f6ceb6eef3dec3cbcc06d2111ea68c/merged major:0 minor:156 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/f1ca4a54f3a89240a2e6a83c0c73f7b79c24b6de3a2e557f74aa927b08c77d21/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/5d84dfa049079416ad52ee102ae46409ad218ea0655c9fd07828c03e46e1a4ea/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d48290ffe906bc74fa545a8045baf19b2552bac821a5a8607806a3c1bf94422c/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/9e2d75696252d4d393a93b5ee6a988a44806b6a896bf2d8b71a8efc2cf714fe2/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/dcc51ee80a2e75e913c34e218774b9e0c03532dad7228c8be27bc5a466fae034/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/81184e0ebdf8516010f307e3f62533af86ab550fd2896d1a6611dce1dff97007/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/e97a6dbe70249c63459a5bc2b7cc7d9f2940cff4f651f8c4f9ff771c7d681d20/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/b6a86868b8a8c6fa89c24cdf21ae7f0e9708c320f9219fc4ad687069ae5ee9a9/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/d3e0044195e25c503655ebd92d331abd9fa9d35b68c7821b250940b2f102310d/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/645533ba54dd86284a8f1484b67f94b2344bd48872a0e94fb1543ac5d85b4f06/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/fac31110ecb172c96ddcae5d45f081bd49ea808193600327e4cc39b0533b79bb/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/66023932ef0c3985ace171dbae807c27bd57fa9b70087ddf531ded9aa82ced58/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/9f7e7d2d4f1dbdbeba1ecf9d18e7adbacc9ef2e60cdc93e8b2bb01fc7489e676/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/75708a156f82f1d24fad2a76cf95e840f3c2ba6ae2ad17878cabce4d1d392d94/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-261:{mountpoint:/var/lib/containers/storage/overlay/505ffba732f71cad8ff110134fed7342672e3d2b93167f7708292218c4954d3d/merged major:0 minor:261 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/dd8108534a69e43df2c069fbab39ac129ed525c4b15f7b109131b3e842945b16/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/9b42c4993bf971cc41fe865435f390017bb51f4f68aa0f3cf92849d83b289a67/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/3ba37e861465f85d32537f0c0a77934482bd391cfb36543b5d7dd65f54528a50/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/bb36e352e31cb15c32cb9b26583cd4c72d71184d80469a1f4f9ec36c38254526/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/95b2022799c47806c6752b76fe43a53e9ff63546df12b03f00832ee510762e2d/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/0d401df038dad97a6ecaa38c6bb64da79126d802119ebdc26d41cf09be6c61c8/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/6ddb156c812c9a15c40888433b66df084ce6ddc5862202d5a579fcd8ba921e29/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/18df8e7172a36f9b1db56795e55d56defd023cf254b5b79d9332b68183e85261/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/547abbc65aade01159ede23d15cfae9b9a0fc88653081374d951e12e2919c733/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/aef8f22bbd248bf2974330582ed36cdd2634a1112c03827bfa08afa836afcc73/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/55540cf06c077087d2554f2582bb90ee2a8561fc995caafc2a17e173a4b2e7f4/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-321:{mountpoint:/var/lib/containers/storage/overlay/e2cbbdf44eb572cb8371afee1f2c7e29558d740b7907dd7ae2ce851e00360fda/merged major:0 minor:321 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/017d99c70ddbd3c745e2631759ef95c27ade31dd3e99e6e52fc1861291e7d40c/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/4659f087670f1e3a4b92b0cc0ede0bcfc6ef8eaf14f208dcff39b587aaf5c394/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/8de7263b161a122305f814148a54beec2adc5a897a992b8609e596f0486060c4/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/052d8645334e14755ec5683ca3a50451ba674ad5950f3f0670da7ce984d3ad18/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/53f9e044c68a899ae74117615736303720aa1422a69f1633cde958d8e1b89839/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/7a4090cb9441701a43909693d2cbca2a282e70d4a2031c4f98dffd55caa060a5/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/330e6f6fa4f393b9af86500d4af386679755984b587a306838be8266e3f5a8aa/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/7d46fb3462bfd72b5b4a20380332434beb8dc9b8f36ae9e2c8feed2edbe3ade5/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/d730e958ec6762afcc9ba50c651a83791a096f5fd7a1aac126baf7272410aa40/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/bf4fe5417a7d79d892b0c75b46eb8c3eca6dc284f4342ef785d3982dc45409ce/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/e811cbebf51b716e54b8239f43da8ae4ebdccba7a5508b2b55c454d103ba79b2/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/9c6746c64b7ea7af029e46883aef7f19023cb04251fceae33e4f60902cc3c003/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/4674b50009a7eddac98b48eaab41b20fd0e06da649117819192b38fb8e7e0ab7/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/0d4257e56d3a754ea1a9c2f93cd2446270ae3a40da1384b7167b1b85a3dee194/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-409:{mountpoint:/var/lib/containers/storage/overlay/ae502992570879cfc84a935f0431d9f69e1b4b0085e837339d8ff94e176e10a5/merged major:0 minor:409 fsType:overlay blockSize:0} overlay_0-411:{mountpoint:/var/lib/containers/storage/overlay/834c50eb21f6bcdb49f0d0259f245e1f640d742d6bcaf3adc21c36bf31856494/merged major:0 minor:411 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/64280affbb9a86e0c0969c92dc28cb62e489358ba83c7162d315ddf100f94577/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-419:{mountpoint:/var/lib/containers/storage/overlay/d4c718124ccca7600ab09ecc6f417ccadcb50a0a9223c88ac0ab3c60f3ec767d/merged major:0 minor:419 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/705ec7550e3b1fa853858a6465f2f802a1d4e4d3b3f12f6100ea41cc1e331c20/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/b59a6251a773a3bf86ca3eb33f76812ca94d21712d4b308c4faccc74593197e4/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/20178dc0f398d3dc8b5a08f4996d4c52da2422e287388fd240e3c63ddba1002a/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/655659f299897ed4602d98d69090d8860631ab6673a5c9764636d43ccfc475aa/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/2f7f4e527729bcb257de46d00891b3657688ad220cc520e44d3c324628b3e2ac/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/c5b9d927848113d0f412b0e8b38263d76480f44cda9884f18118b2c30b65d68d/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/8bb5d9877c548395df9a7fb8dd50d7bcc36c24cfdcd7d49a436b03daea882f59/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/20ced8540084efeeec03473e5210274459f674f51711af89d67ec49f03aa0282/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/b2b5b299a22a681d8f9de56c286b5e0c6b15a6733ced79bc897c356172aa45ca/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/ac677b01fbbb52c810c003b068c0bea37e9492d8fd666072df09494f21d53754/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/429dfeaa385268cab69cb39f6512118aa0dd17fb3031ae12b6e7f6f4a558c6c4/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.861652    6389 manager.go:217] Machine: {Timestamp:2026-10-30 20:04:39.861198885 +0000 UTC m=+0.060771805 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103257 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f965da3a7fedd028695de741d739c SystemUUID:ec2f965d-a3a7-fedd-0286-95de741d739c BootID:ff2ab5e3-9843-4f88-8a75-3c5668d0a397 Filesystems:[{Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:249 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41/volumes/kubernetes.io~projected/kube-api-access-t8gkt DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a40bd710d363b612693bc3c34aa642eaae991f884783c59ef78f254aecbddd48/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8d6d749-b397-476f-83ca-774cc1aa5a0d/volumes/kubernetes.io~projected/kube-api-access-p9fnl DeviceMajor:0 DeviceMinor:341 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:281 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5efb380a-6530-4db0-88ec-2d59891344f1/volumes/kubernetes.io~projected/kube-api-access-dk2l2 DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:224 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:219 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2a42101116e74f5179690dc4552ae786af5db9987483e3f52a50bccf73ed899c/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:284 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b8b667bd7fc2ec60bf6b3b17417832b5c074d8c777184bb22d66531ef183543e/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:281 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:283 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:280 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-156 DeviceMajor:0 DeviceMinor:156 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~projected/kube-api-access-w95dh DeviceMajor:0 DeviceMinor:254 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-261 DeviceMajor:0 DeviceMinor:261 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ebe215c4adaed7cb22d4675700c7d9e6b05388076638d1250dd29aa4629b07da/userdata/shm DeviceMajor:0 DeviceMinor:255 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:247 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/de259d4b-08fb-48bb-9aad-0a381205090f/volumes/kubernetes.io~projected/kube-api-access-s7wjj DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bf6032baf294e242c09d5ecc9c5f336f90539da9c1b3024538545e0a8fdea91e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-153 DeviceMajor:0 DeviceMinor:153 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab48927b-5485-4685-84d1-6141d4de8ceb/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b/userdata/shm DeviceMajor:0 DeviceMinor:375 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e81014bb3c9331af9e5f24ffbc5149e96ebd9e88288f82458ee86023ac722868/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:282 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:369 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:373 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4ea62f92-610f-4211-9153-474cd90cadaa/volumes/kubernetes.io~projected/kube-api-access-q59hq DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3ce05b8e-185a-48d6-b18b-5858b2959885/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:392 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4ea62f92-610f-4211-9153-474cd90cadaa/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:397 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-155 DeviceMajor:0 DeviceMinor:155 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/084a4bf4c52fba5437b07b81b4d54d52a9435465a0869bff2e4a34e433254644/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ab48927b-5485-4685-84d1-6141d4de8ceb/volumes/kubernetes.io~projected/kube-api-access-bprnz DeviceMajor:0 DeviceMinor:203 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/dd239462-09eb-4071-83f6-171abe1cd346/volumes/kubernetes.io~projected/kube-api-access-wxq75 DeviceMajor:0 DeviceMinor:204 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:283 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-411 DeviceMajor:0 DeviceMinor:411 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b5ddf8fa-c589-4ef9-8299-83ac15845b10/volumes/kubernetes.io~projected/kube-api-access-pc8vv DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ba5acb1d34bfecd6f8dabbfd56e626f341896b94f1e00fa9e7651c5f93affb1/userdata/shm DeviceMajor:0 DeviceMinor:257 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:283 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ce83785c43575b988dfc59ee502446025dde26b6a67fe37d3a4e1784bc14e53/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-409 DeviceMajor:0 DeviceMinor:409 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/028c4f21523f204ba6936780bd1f189b810ac6567108824a205b26069c87fe89/userdata/shm DeviceMajor:0 DeviceMinor:97 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5421f3d7d8b7165980d39980e605add980e3701cf85cb44824efba1f37400d70/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~projected/kube-api-access-vsn4c DeviceMajor:0 DeviceMinor:253 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/969b45671329d3489792159766a2f03cb866b89355e27ae589561e18fd498454/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef052b89e2656799213f40f324d1a3101cfc6ca0b89a201d67c57260ff808915/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aef9d7d239db9b1b97665dc456467d15d542598e5107f30ec23d7f35ff8d3fde/userdata/shm DeviceMajor:0 DeviceMinor:400 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/eab05532-1d94-4218-bef2-a7f43bfd25e8/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:248 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~projected/kube-api-access-ftc9t DeviceMajor:0 DeviceMinor:252 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/14516aae76067e4c151654e337dd0489eca0b19d4a07fbbf43baa3334ab814e9/userdata/shm DeviceMajor:0 DeviceMinor:405 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/85cc4235496e2f6cc384db1909747d0e2e67bddb052a82e302f580f1a4763711/userdata/shm DeviceMajor:0 DeviceMinor:259 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:279 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81a971d1-9ef7-43cd-9f79-72ff95212079/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/81a971d1-9ef7-43cd-9f79-72ff95212079/volumes/kubernetes.io~projected/kube-api-access-7kpmf DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes/kubernetes.io~projected/kube-api-access-5fzc7 DeviceMajor:0 DeviceMinor:225 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dd239462-09eb-4071-83f6-171abe1cd346/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-321 DeviceMajor:0 DeviceMinor:321 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8d6d749-b397-476f-83ca-774cc1aa5a0d/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:337 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8bdc9936-09aa-4583-95e7-499428c2b9b0/volumes/kubernetes.io~projected/kube-api-access-sdc9g DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6ba6f137-f70b-41a4-9e2b-871bb10b4f07/volumes/kubernetes.io~projected/kube-api-access-7tr6l DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:250 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:286 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~projected/kube-api-access-xtshj DeviceMajor:0 DeviceMinor:287 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7e778f2cfe712315da91830d48b91328f9febbdfeb8c50c4efa6d7af0db50d7d/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/23c7a152-2247-4d92-9a1e-8f6a85b4954e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:243 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:283 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes/kubernetes.io~projected/kube-api-access-z2fzg DeviceMajor:0 DeviceMinor:374 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/f4ce54b1-7691-41ff-9a3e-56b99c04ced4/volumes/kubernetes.io~projected/kube-api-access-tgrsp DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:285 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/26dcecb1-c6af-49e9-a840-b7197582b0cb/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:251 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/23655d1273bceb9573acf489f51ddba6d5c5c661b860474176a0f7aa00d346fe/userdata/shm DeviceMajor:0 DeviceMinor:398 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ed30ac6f-8649-4036-963c-25bef426479d/volumes/kubernetes.io~projected/kube-api-access-b78wd DeviceMajor:0 DeviceMinor:402 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:275 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-419 DeviceMajor:0 DeviceMinor:419 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/cdaa2197-530d-48e5-899c-5ccc58f4016a/volumes/kubernetes.io~projected/kube-api-access-zzlpm DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:14516aae76067e4 MacAddress:1a:82:9c:88:17:86 Speed:10000 Mtu:8901} {Name:23655d1273bceb9 MacAddress:3a:93:d2:51:2b:9f Speed:10000 Mtu:8901} {Name:2a42101116e74f5 MacAddress:42:01:c7:9a:f7:d5 Speed:10000 Mtu:8901} {Name:55d5d80ed4363d8 MacAddress:62:b9:af:af:8b:e4 Speed:10000 Mtu:8901} {Name:5ba5acb1d34bfec MacAddress:ce:a3:90:03:ca:0e Speed:10000 Mtu:8901} {Name:7e778f2cfe71231 MacAddress:36:5f:fd:d7:45:e9 Speed:10000 Mtu:8901} {Name:aef9d7d239db9b1 MacAddress:de:1b:0f:3a:12:b7 Speed:10000 Mtu:8901} {Name:b8b667bd7fc2ec6 MacAddress:3a:47:28:a1:30:f3 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ef:54:8c:c3 Speed:0 Mtu:9001} {Name:br-int MacAddress:32:41:37:58:19:1e Speed:0 Mtu:8901} {Name:e19d5cf70711431 MacAddress:12:ad:5d:fe:ca:be Speed:10000 Mtu:8901} {Name:ebe215c4adaed7c MacAddress:8e:12:ec:ba:39:6a Speed:10000 Mtu:8901} {Name:ef052b89e265679 MacAddress:1a:82:f8:fd:04:42 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ef:54:8c:c3 Speed:0 Mtu:9001} {Name:f3e7e2d4e1e5d14 MacAddress:aa:2e:df:ff:15:34 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:7a:e8:99:82:77:6e Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:6e:1b:38:dc:60:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862000    6389 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862041    6389 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862374    6389 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862392    6389 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-14.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862524    6389 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862532    6389 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862545    6389 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862559    6389 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862630    6389 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862824    6389 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862928    6389 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862960    6389 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862970    6389 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.862983    6389 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.863021    6389 kubelet.go:405] "Adding apiserver pod source"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.863041    6389 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.863975    6389 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 20:04:39.864046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.863986    6389 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:04:39.864789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.864476    6389 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 20:04:39.868150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868098    6389 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 20:04:39.868150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868112    6389 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:04:39.868261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868227    6389 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 20:04:39.868422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868409    6389 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 20:04:39.868455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868439    6389 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 20:04:39.868534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868525    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 20:04:39.868563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868540    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 20:04:39.868563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868549    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 20:04:39.868563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868558    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868567    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868574    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868579    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868585    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868592    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868597    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868612    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868622    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 20:04:39.868643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868645    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 20:04:39.868880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.868655    6389 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 20:04:39.872229 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.872217    6389 server.go:1263] "Started kubelet"
Oct 30 20:04:39.872869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.872302    6389 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 20:04:39.872869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.872354    6389 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 20:04:39.872869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.872434    6389 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 20:04:39.872869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.872726    6389 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 20:04:39.872738 ip-10-0-129-14 systemd[1]: Started Kubernetes Kubelet.
Oct 30 20:04:39.873641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.873624    6389 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 20:04:39.876201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876184    6389 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876197    6389 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876269    6389 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:58:30 +0000 UTC" deadline="2028-05-14 19:15:34.260555458 +0000 UTC"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876320    6389 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13487h10m54.384238694s"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876345    6389 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876356    6389 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.876418    6389 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 20:04:39.878857 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:39.877071    6389 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-14.us-east-2.compute.internal\" not found"
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880580    6389 factory.go:55] Registering systemd factory
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880602    6389 factory.go:223] Registration of the systemd container factory successfully
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880775    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3de1609a-ab4c-448c-bc7f-fcdbcc70ff41" volumeName="kubernetes.io/projected/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-kube-api-access-t8gkt" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880818    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880835    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880857    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880870    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880887    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5ddf8fa-c589-4ef9-8299-83ac15845b10" volumeName="kubernetes.io/configmap/b5ddf8fa-c589-4ef9-8299-83ac15845b10-serviceca" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880900    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880914    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880943    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e" volumeName="kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880957    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e" volumeName="kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880976    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab48927b-5485-4685-84d1-6141d4de8ceb" volumeName="kubernetes.io/secret/ab48927b-5485-4685-84d1-6141d4de8ceb-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.880990    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dd239462-09eb-4071-83f6-171abe1cd346" volumeName="kubernetes.io/configmap/dd239462-09eb-4071-83f6-171abe1cd346-config-volume" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881006    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dd239462-09eb-4071-83f6-171abe1cd346" volumeName="kubernetes.io/empty-dir/dd239462-09eb-4071-83f6-171abe1cd346-tmp-dir" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881017    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eab05532-1d94-4218-bef2-a7f43bfd25e8" volumeName="kubernetes.io/secret/eab05532-1d94-4218-bef2-a7f43bfd25e8-agent-certs" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881045    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ea62f92-610f-4211-9153-474cd90cadaa" volumeName="kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881061    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" volumeName="kubernetes.io/projected/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-kube-api-access-7tr6l" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881073    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdaa2197-530d-48e5-899c-5ccc58f4016a" volumeName="kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881090    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c7a152-2247-4d92-9a1e-8f6a85b4954e" volumeName="kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881107    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881141    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ce54b1-7691-41ff-9a3e-56b99c04ced4" volumeName="kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-tmp" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881156    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881173    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" volumeName="kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-binary-copy" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881194    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdaa2197-530d-48e5-899c-5ccc58f4016a" volumeName="kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-config" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881210    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de259d4b-08fb-48bb-9aad-0a381205090f" volumeName="kubernetes.io/configmap/de259d4b-08fb-48bb-9aad-0a381205090f-iptables-alerter-script" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881232    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de259d4b-08fb-48bb-9aad-0a381205090f" volumeName="kubernetes.io/projected/de259d4b-08fb-48bb-9aad-0a381205090f-kube-api-access-s7wjj" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881249    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26dcecb1-c6af-49e9-a840-b7197582b0cb" volumeName="kubernetes.io/projected/26dcecb1-c6af-49e9-a840-b7197582b0cb-kube-api-access-ftc9t" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881265    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp" seLinuxMountContext=""
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881285    6389 factory.go:153] Registering CRI-O factory
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881300    6389 factory.go:223] Registration of the crio container factory successfully
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881368    6389 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881388    6389 factory.go:103] Registering Raw factory
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881406    6389 manager.go:1199] Started watching for new ooms in manager
Oct 30 20:04:39.881701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881283    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e" volumeName="kubernetes.io/projected/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-api-access-w95dh" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881756    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881796    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881810    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ce54b1-7691-41ff-9a3e-56b99c04ced4" volumeName="kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-tuned" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881821    6389 manager.go:319] Starting recovery of all containers
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881824    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881843    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c7a152-2247-4d92-9a1e-8f6a85b4954e" volumeName="kubernetes.io/projected/23c7a152-2247-4d92-9a1e-8f6a85b4954e-kube-api-access-vsn4c" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881856    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26dcecb1-c6af-49e9-a840-b7197582b0cb" volumeName="kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-metrics-client-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881874    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26dcecb1-c6af-49e9-a840-b7197582b0cb" volumeName="kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-tls" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881886    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881900    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e" volumeName="kubernetes.io/empty-dir/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-volume-directive-shadow" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881917    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" volumeName="kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881930    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8d6d749-b397-476f-83ca-774cc1aa5a0d" volumeName="kubernetes.io/secret/a8d6d749-b397-476f-83ca-774cc1aa5a0d-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881947    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8d6d749-b397-476f-83ca-774cc1aa5a0d" volumeName="kubernetes.io/projected/a8d6d749-b397-476f-83ca-774cc1aa5a0d-kube-api-access-p9fnl" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881959    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab48927b-5485-4685-84d1-6141d4de8ceb" volumeName="kubernetes.io/configmap/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881973    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ce05b8e-185a-48d6-b18b-5858b2959885" volumeName="kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.881990    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882003    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882021    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5efb380a-6530-4db0-88ec-2d59891344f1" volumeName="kubernetes.io/empty-dir/5efb380a-6530-4db0-88ec-2d59891344f1-tmp-dir" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882034    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5ddf8fa-c589-4ef9-8299-83ac15845b10" volumeName="kubernetes.io/projected/b5ddf8fa-c589-4ef9-8299-83ac15845b10-kube-api-access-pc8vv" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882050    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ea62f92-610f-4211-9153-474cd90cadaa" volumeName="kubernetes.io/projected/4ea62f92-610f-4211-9153-474cd90cadaa-kube-api-access-q59hq" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882065    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882077    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882094    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882107    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882168    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdaa2197-530d-48e5-899c-5ccc58f4016a" volumeName="kubernetes.io/secret/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882182    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c7a152-2247-4d92-9a1e-8f6a85b4954e" volumeName="kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882194    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882212    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed30ac6f-8649-4036-963c-25bef426479d" volumeName="kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882224    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c7a152-2247-4d92-9a1e-8f6a85b4954e" volumeName="kubernetes.io/configmap/23c7a152-2247-4d92-9a1e-8f6a85b4954e-metrics-client-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882267    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26dcecb1-c6af-49e9-a840-b7197582b0cb" volumeName="kubernetes.io/empty-dir/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-textfile" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882283    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5efb380a-6530-4db0-88ec-2d59891344f1" volumeName="kubernetes.io/projected/5efb380a-6530-4db0-88ec-2d59891344f1-kube-api-access-dk2l2" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882294    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e" volumeName="kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-metrics-client-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882312    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bdc9936-09aa-4583-95e7-499428c2b9b0" volumeName="kubernetes.io/projected/8bdc9936-09aa-4583-95e7-499428c2b9b0-kube-api-access-sdc9g" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882327    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882340    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dd239462-09eb-4071-83f6-171abe1cd346" volumeName="kubernetes.io/secret/dd239462-09eb-4071-83f6-171abe1cd346-metrics-tls" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882360    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eab05532-1d94-4218-bef2-a7f43bfd25e8" volumeName="kubernetes.io/configmap/eab05532-1d94-4218-bef2-a7f43bfd25e8-konnectivity-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882373    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882391    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26dcecb1-c6af-49e9-a840-b7197582b0cb" volumeName="kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882403    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882417    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882437    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3de1609a-ab4c-448c-bc7f-fcdbcc70ff41" volumeName="kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-daemon-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882451    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab48927b-5485-4685-84d1-6141d4de8ceb" volumeName="kubernetes.io/projected/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-api-access-bprnz" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882468    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdaa2197-530d-48e5-899c-5ccc58f4016a" volumeName="kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-env-overrides" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882481    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dd239462-09eb-4071-83f6-171abe1cd346" volumeName="kubernetes.io/projected/dd239462-09eb-4071-83f6-171abe1cd346-kube-api-access-wxq75" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882496    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51cefd7f-e086-40ff-b71b-c796ea9489e2" volumeName="kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882517    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" volumeName="kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882530    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882548    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab48927b-5485-4685-84d1-6141d4de8ceb" volumeName="kubernetes.io/empty-dir/ab48927b-5485-4685-84d1-6141d4de8ceb-data-volume" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882563    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac7a756f-0197-4cfb-bb53-dfde71285608" volumeName="kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882582    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdaa2197-530d-48e5-899c-5ccc58f4016a" volumeName="kubernetes.io/projected/cdaa2197-530d-48e5-899c-5ccc58f4016a-kube-api-access-zzlpm" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882594    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81a971d1-9ef7-43cd-9f79-72ff95212079" volumeName="kubernetes.io/secret/81a971d1-9ef7-43cd-9f79-72ff95212079-cert" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882607    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26dcecb1-c6af-49e9-a840-b7197582b0cb" volumeName="kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882624    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882642    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e" volumeName="kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882660    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81a971d1-9ef7-43cd-9f79-72ff95212079" volumeName="kubernetes.io/projected/81a971d1-9ef7-43cd-9f79-72ff95212079-kube-api-access-7kpmf" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882708    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ce54b1-7691-41ff-9a3e-56b99c04ced4" volumeName="kubernetes.io/projected/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-kube-api-access-tgrsp" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882725    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a590085-c2de-41b6-aaff-a65de1a3a1c9" volumeName="kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882746    6389 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3de1609a-ab4c-448c-bc7f-fcdbcc70ff41" volumeName="kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cni-binary-copy" seLinuxMountContext=""
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882759    6389 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 20:04:39.883404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.882767    6389 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 20:04:39.886276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.886255    6389 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:04:39.887234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.886523    6389 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:04:39.890181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.889622    6389 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 20:04:39.890181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.889899    6389 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:04:39.903760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.903735    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:04:39.904967 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.904775    6389 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 20:04:39.906327 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.906305    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:04:39.906542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.906519    6389 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 20:04:39.906542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.906542    6389 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 20:04:39.906717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.906565    6389 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 20:04:39.906717 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:39.906607    6389 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 20:04:39.910910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.910890    6389 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:04:39.929569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.929542    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:04:39.929920 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.929882    6389 generic.go:356] "Generic (PLEG): container finished" podID="cdaa2197-530d-48e5-899c-5ccc58f4016a" containerID="1cec20fec7905162490d7ff0b1420979b6a2d6d2b80a98eb66677e509a427f58" exitCode=1
Oct 30 20:04:39.939233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.939210    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74" exitCode=0
Oct 30 20:04:39.941995 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.941960    6389 generic.go:356] "Generic (PLEG): container finished" podID="26dcecb1-c6af-49e9-a840-b7197582b0cb" containerID="c53df75307c0a5c1d353deb127f9768933620b3367725e29987d168073de35e7" exitCode=0
Oct 30 20:04:39.961010 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.960978    6389 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="90e740d66308ef7bf5ce5e7ab45b7e88633dbf64cff204c380e56e00b847927f" exitCode=0
Oct 30 20:04:39.961010 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.961008    6389 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="0197a3e51ca3a6e60081a339a0b24ebb9eeb5427c85d65dede1f540961a75229" exitCode=0
Oct 30 20:04:39.961167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.961018    6389 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="007d6ecc5797bba0a63ebc82bf25bfd3a203b875b448ae19dc3e4f14553b0e03" exitCode=0
Oct 30 20:04:39.961167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.961027    6389 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="045becbc89c720f021ae1ec601f7af4586a4a693ffe009d56f5a66f707649dde" exitCode=0
Oct 30 20:04:39.961167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.961036    6389 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="5eb78c901575be229432422e8c81b792c7d890f11c3dca363bc78cc9a7ab5e7b" exitCode=0
Oct 30 20:04:39.961167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.961044    6389 generic.go:356] "Generic (PLEG): container finished" podID="6ba6f137-f70b-41a4-9e2b-871bb10b4f07" containerID="824d7745658ac6bd7ae58d98b1eb24c73fa7162e78d253ab3dd31c24e49e1818" exitCode=0
Oct 30 20:04:39.964343 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.964319    6389 generic.go:356] "Generic (PLEG): container finished" podID="1e91bd056ef60d099398ca83c4eeeeab" containerID="4aad6c6a185c7ede00a6644377cbeee5f226de796ed3e136fba74e66da506540" exitCode=0
Oct 30 20:04:39.989535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:39.989516    6389 manager.go:324] Recovery completed
Oct 30 20:04:40.006951 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.006923    6389 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 20:04:40.012038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.011981    6389 cpu_manager.go:226] "Starting" policy="none"
Oct 30 20:04:40.012038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.011995    6389 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 20:04:40.012038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012012    6389 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012149    6389 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012160    6389 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012178    6389 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012184    6389 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012189    6389 policy_none.go:50] "Start"
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012198    6389 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 20:04:40.012211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.012207    6389 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 20:04:40.013264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.013250    6389 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 20:04:40.013298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.013264    6389 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 20:04:40.013298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.013270    6389 policy_none.go:44] "Start"
Oct 30 20:04:40.017932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.017903    6389 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 20:04:40.020649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.020636    6389 manager.go:341] "Starting Device Plugin manager"
Oct 30 20:04:40.021192 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.020663    6389 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 20:04:40.021289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021195    6389 server.go:87] "Starting device plugin registration server"
Oct 30 20:04:40.021452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021441    6389 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 20:04:40.021502 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021453    6389 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 20:04:40.021553 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021524    6389 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 20:04:40.021637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021610    6389 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 20:04:40.021675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021648    6389 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 20:04:40.021675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021656    6389 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 20:04:40.021768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.021695    6389 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T20:04:40.02163438Z","UUID":"2c9fd55f-f8d5-4a6c-9bc5-19ef05dedce0","Handler":null,"Name":"","Endpoint":""}
Oct 30 20:04:40.023081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.023059    6389 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 20:04:40.023223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.023102    6389 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 20:04:40.023332 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.023307    6389 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 20:04:40.122239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.122213    6389 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:04:40.124324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.124308    6389 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:04:40.124391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.124337    6389 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:04:40.124391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.124348    6389 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-14.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:04:40.124391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.124387    6389 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.137223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.137204    6389 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.137295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.137257    6389 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.207979 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.207914    6389 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"]
Oct 30 20:04:40.208634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208608    6389 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.208763 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208589    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"bfb87cbb02f3462c0bb6e1279ac3fe5a","Type":"ContainerStarted","Data":"e7abda52eed75aae252331091b5b802aad74d0d2165cb174d4e1989888cba360"}
Oct 30 20:04:40.208763 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208744    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"bfb87cbb02f3462c0bb6e1279ac3fe5a","Type":"ContainerStarted","Data":"ed1535221ab92947dbf3e007f90f6da8c9ecd8e6408a2cca46d3dbaf215628ea"}
Oct 30 20:04:40.208851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208814    6389 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.208881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208868    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"1e91bd056ef60d099398ca83c4eeeeab","Type":"ContainerStarted","Data":"29e6574cbf806c0e8419c3137c02924f7e3edba1f88ac0f7ee5c0f7a2bc8b129"}
Oct 30 20:04:40.208915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208891    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"1e91bd056ef60d099398ca83c4eeeeab","Type":"ContainerDied","Data":"4aad6c6a185c7ede00a6644377cbeee5f226de796ed3e136fba74e66da506540"}
Oct 30 20:04:40.208944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.208912    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal" event={"ID":"1e91bd056ef60d099398ca83c4eeeeab","Type":"ContainerStarted","Data":"743e509b42ca7f8a99272825a0c4e62b092f05e1cc8ce5ff9802d746e58950ac"}
Oct 30 20:04:40.221194 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.221177    6389 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:04:40.221279 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.221225    6389 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.222355 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.222343    6389 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:04:40.222408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.222372    6389 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.291727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.291657    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.291727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.291719    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.291852 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.291747    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfb87cbb02f3462c0bb6e1279ac3fe5a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"bfb87cbb02f3462c0bb6e1279ac3fe5a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.392311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.392275    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.392311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.392313    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.392516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.392331    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfb87cbb02f3462c0bb6e1279ac3fe5a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"bfb87cbb02f3462c0bb6e1279ac3fe5a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.392516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.392374    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfb87cbb02f3462c0bb6e1279ac3fe5a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"bfb87cbb02f3462c0bb6e1279ac3fe5a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.392516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.392401    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.392516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.392401    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e91bd056ef60d099398ca83c4eeeeab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" (UID: \"1e91bd056ef60d099398ca83c4eeeeab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.864085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.864050    6389 apiserver.go:52] "Watching apiserver"
Oct 30 20:04:40.875095 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.875065    6389 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 20:04:40.875649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.875629    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-6whf2","openshift-monitoring/alertmanager-main-0","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn","openshift-console/console-7ddc794486-fvllw","openshift-image-registry/node-ca-zhts8","openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw","openshift-network-diagnostics/network-check-target-zv26m","kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal","openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w","kube-system/global-pull-secret-syncer-sgpdx","openshift-cluster-node-tuning-operator/tuned-fbmvs","openshift-dns/node-resolver-mzzv9","openshift-ingress-canary/ingress-canary-ww6bn","openshift-multus/multus-additional-cni-plugins-49thc","openshift-console/console-69d97cbb97-zgnfn","openshift-dns/dns-default-t7h8t","openshift-insights/insights-runtime-extractor-nnnfl","openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l","openshift-monitoring/node-exporter-d6vx6","kube-system/konnectivity-agent-9jh8v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-jnfz5","openshift-multus/multus-8lgs5","openshift-multus/network-metrics-daemon-c9kfm"]
Oct 30 20:04:40.880640 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.880603    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.881057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.881032    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9mzsl\""
Oct 30 20:04:40.881246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.881184    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 20:04:40.881322 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.881282    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rlckg\""
Oct 30 20:04:40.881376 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.881337    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.881376 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.881208    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.881687 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.881658    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.882303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.882242    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.882593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.882245    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 20:04:40.882673 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.882658    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.882816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.882792    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hms2c\""
Oct 30 20:04:40.882956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.882937    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-phqsm\""
Oct 30 20:04:40.883066 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.883012    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.883171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.883154    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 20:04:40.883443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.883426    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 20:04:40.883580 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.883566    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.884165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884147    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 20:04:40.884259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884236    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.884259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884248    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 20:04:40.884259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884260    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.884448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884406    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-26gph\""
Oct 30 20:04:40.884751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884737    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 20:04:40.884974 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.884958    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 20:04:40.885051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.885018    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hntbg\""
Oct 30 20:04:40.885135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.885097    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 20:04:40.885135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.885130    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.885440 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.885427    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.886057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.886033    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 20:04:40.886171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.886153    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-84sqw\""
Oct 30 20:04:40.886307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.886288    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 20:04:40.886832 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.886788    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 20:04:40.887078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.887057    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-cfnmr\""
Oct 30 20:04:40.887169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.887086    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 20:04:40.887232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.887193    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pxq5r\""
Oct 30 20:04:40.887232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.887211    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 20:04:40.887856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.887826    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qkmrx\""
Oct 30 20:04:40.887986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.887971    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 20:04:40.888267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888246    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.888583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888422    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rnw2v\""
Oct 30 20:04:40.888583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888437    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.888583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888456    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rx62b\""
Oct 30 20:04:40.888767 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888693    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.888817 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888780    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.888990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.888952    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 20:04:40.889209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.889190    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.889291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.889228    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-nfbvj\""
Oct 30 20:04:40.889685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.889668    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 20:04:40.889761 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.889740    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 20:04:40.889809 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.889794    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c2z8r\""
Oct 30 20:04:40.889866 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.889840    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.890039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.890026    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 20:04:40.890621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.890424    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 20:04:40.890760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.890744    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.890832 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.890797    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.890887 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.890860    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j5tm5\""
Oct 30 20:04:40.891658 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.891635    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 20:04:40.891782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.891765    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcdxj\""
Oct 30 20:04:40.891965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.891950    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 20:04:40.892881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.892865    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 20:04:40.893915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.893896    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de259d4b-08fb-48bb-9aad-0a381205090f-host-slash\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.893963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.893928    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.893963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.893955    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-socket-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.894024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894002    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdc9g\" (UniqueName: \"kubernetes.io/projected/8bdc9936-09aa-4583-95e7-499428c2b9b0-kube-api-access-sdc9g\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.894065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894045    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:40.894201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894182    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7wjj\" (UniqueName: \"kubernetes.io/projected/de259d4b-08fb-48bb-9aad-0a381205090f-kube-api-access-s7wjj\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.894253 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894217    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-registration-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.894307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894255    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-device-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.894307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894280    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.894307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894303    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-sys-fs\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.894488 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894338    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de259d4b-08fb-48bb-9aad-0a381205090f-iptables-alerter-script\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.894571 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.894552    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de259d4b-08fb-48bb-9aad-0a381205090f-iptables-alerter-script\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.905177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.905158    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdc9g\" (UniqueName: \"kubernetes.io/projected/8bdc9936-09aa-4583-95e7-499428c2b9b0-kube-api-access-sdc9g\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.905731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.905348    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7wjj\" (UniqueName: \"kubernetes.io/projected/de259d4b-08fb-48bb-9aad-0a381205090f-kube-api-access-s7wjj\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.905731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.905645    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b78wd\" (UniqueName: \"kubernetes.io/projected/ed30ac6f-8649-4036-963c-25bef426479d-kube-api-access-b78wd\") pod \"network-check-target-zv26m\" (UID: \"ed30ac6f-8649-4036-963c-25bef426479d\") " pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:40.971544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.971517    6389 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.971696 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.971628    6389 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.979297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.979269    6389 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 20:04:40.980686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.980670    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 20:04:40.981015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.980989    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 20:04:40.981138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981039    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 20:04:40.981138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981050    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 20:04:40.981243 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981226    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9wn4n\""
Oct 30 20:04:40.981326 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981282    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 20:04:40.981326 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981317    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 20:04:40.981428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981351    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fg55m\""
Oct 30 20:04:40.981428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981371    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 20:04:40.981428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981381    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 20:04:40.981428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981415    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 20:04:40.981607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981438    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kpzzd\""
Oct 30 20:04:40.981607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981448    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 20:04:40.981607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981491    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 20:04:40.981607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981531    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 20:04:40.981607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981603    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 20:04:40.981781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981669    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 20:04:40.981781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981733    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8jpfj\""
Oct 30 20:04:40.981781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981767    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 20:04:40.981861 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981799    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 20:04:40.981861 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.981847    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 20:04:40.983632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.983614    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 20:04:40.984218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.984205    6389 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:04:40.984272 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.984250    6389 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.984867 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.984853    6389 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:04:40.984906 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.984887    6389 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-14.us-east-2.compute.internal"
Oct 30 20:04:40.994837 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994808    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.994898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994848    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95dh\" (UniqueName: \"kubernetes.io/projected/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-api-access-w95dh\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:40.994898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994885    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab48927b-5485-4685-84d1-6141d4de8ceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:40.994999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994921    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-k8s-cni-cncf-io\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.994999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994939    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-slash\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.994999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994954    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-systemd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.994999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.994975    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995034    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-env-overrides\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995140    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrsp\" (UniqueName: \"kubernetes.io/projected/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-kube-api-access-tgrsp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.995211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995163    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-textfile\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.995211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995201    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd239462-09eb-4071-83f6-171abe1cd346-config-volume\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995228    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-textfile\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995234    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995236    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab48927b-5485-4685-84d1-6141d4de8ceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995266    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995225    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-env-overrides\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995303    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eab05532-1d94-4218-bef2-a7f43bfd25e8-konnectivity-ca\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995329    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-modprobe-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995356    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995383    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995411    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-sys-fs\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995409    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd239462-09eb-4071-83f6-171abe1cd346-config-volume\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995438    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:40.995464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995446    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81a971d1-9ef7-43cd-9f79-72ff95212079-cert\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995505    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995526    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-kubelet\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995542    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.495519568 +0000 UTC m=+1.695092496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995554    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eab05532-1d94-4218-bef2-a7f43bfd25e8-konnectivity-ca\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995570    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.495555588 +0000 UTC m=+1.695128514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995554    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995506    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-sys-fs\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995590    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995617    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995629    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995636    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995657    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995684    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-run\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995662    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81a971d1-9ef7-43cd-9f79-72ff95212079-cert\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995694    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995698    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.495683445 +0000 UTC m=+1.695256360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995722    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-script-lib\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.995754    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.495727524 +0000 UTC m=+1.695300451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995784    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpmf\" (UniqueName: \"kubernetes.io/projected/81a971d1-9ef7-43cd-9f79-72ff95212079-kube-api-access-7kpmf\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995812    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-bin\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995839    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8d6d749-b397-476f-83ca-774cc1aa5a0d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995863    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk2l2\" (UniqueName: \"kubernetes.io/projected/5efb380a-6530-4db0-88ec-2d59891344f1-kube-api-access-dk2l2\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995873    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-script-lib\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995887    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995911    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cnibin\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.995935    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996002    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-host\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.995999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996019    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzlpm\" (UniqueName: \"kubernetes.io/projected/cdaa2197-530d-48e5-899c-5ccc58f4016a-kube-api-access-zzlpm\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996034    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8d6d749-b397-476f-83ca-774cc1aa5a0d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996039    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd239462-09eb-4071-83f6-171abe1cd346-metrics-tls\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996094    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxq75\" (UniqueName: \"kubernetes.io/projected/dd239462-09eb-4071-83f6-171abe1cd346-kube-api-access-wxq75\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996142    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996186    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996289    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd239462-09eb-4071-83f6-171abe1cd346-metrics-tls\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996362    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bprnz\" (UniqueName: \"kubernetes.io/projected/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-api-access-bprnz\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996463    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-multus\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996465    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996548    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-hostroot\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996583    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996601    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-var-lib-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996621    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eab05532-1d94-4218-bef2-a7f43bfd25e8-agent-certs\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996646    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996666    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysconfig\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996724    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-conf\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996762    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9fnl\" (UniqueName: \"kubernetes.io/projected/a8d6d749-b397-476f-83ca-774cc1aa5a0d-kube-api-access-p9fnl\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996791    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-dbus\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996842    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-config\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996857    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eab05532-1d94-4218-bef2-a7f43bfd25e8-agent-certs\") pod \"konnectivity-agent-9jh8v\" (UID: \"eab05532-1d94-4218-bef2-a7f43bfd25e8\") " pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996860    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996898    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996933    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-systemd\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.996981    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-kubelet\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.997003    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997015    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997042    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovnkube-config\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997044    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.997070    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.497055661 +0000 UTC m=+1.696628582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.997096    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997104    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997152    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.997168    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.497154 +0000 UTC m=+1.696726910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997197    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftc9t\" (UniqueName: \"kubernetes.io/projected/26dcecb1-c6af-49e9-a840-b7197582b0cb-kube-api-access-ftc9t\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997234    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.997242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997259    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de259d4b-08fb-48bb-9aad-0a381205090f-host-slash\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997281    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-socket-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997336    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de259d4b-08fb-48bb-9aad-0a381205090f-host-slash\") pod \"iptables-alerter-6whf2\" (UID: \"de259d4b-08fb-48bb-9aad-0a381205090f\") " pod="openshift-network-operator/iptables-alerter-6whf2"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997346    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997340    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997313    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997372    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-lib-modules\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997410    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-tmp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997401    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-socket-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997460    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-multus-certs\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997466    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997477    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-tmp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997499    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5ddf8fa-c589-4ef9-8299-83ac15845b10-host\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997519    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-systemd-units\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997547    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997595    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q59hq\" (UniqueName: \"kubernetes.io/projected/4ea62f92-610f-4211-9153-474cd90cadaa-kube-api-access-q59hq\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997646    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997659    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997681    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-socket-dir-parent\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997733    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-etc-kubernetes\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997766    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-netns\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997803    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997803    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-etc-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997857    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-ovn\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997889    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-netd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997909    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab48927b-5485-4685-84d1-6141d4de8ceb-crio-socket\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997932    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-sys\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.997972    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-system-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998000    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-netns\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998015    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5ddf8fa-c589-4ef9-8299-83ac15845b10-serviceca\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998030    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-node-log\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998065    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998094    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovn-node-metrics-cert\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998131    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-registration-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998171    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.998181    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998197    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998222    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998220    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-registration-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.998230    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.498217488 +0000 UTC m=+1.697790402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.998256    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.998259    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.998321    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.498312822 +0000 UTC m=+1.697885728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998339    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998357    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998374    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998396    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-metrics-client-ca\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.998409    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.498395704 +0000 UTC m=+1.697968614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998438    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998456    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdaa2197-530d-48e5-899c-5ccc58f4016a-ovn-node-metrics-cert\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998469    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998468    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tr6l\" (UniqueName: \"kubernetes.io/projected/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-kube-api-access-7tr6l\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998499    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998528    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-root\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998555    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998559    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998575    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-metrics-client-ca\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998585    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cnibin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998597    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:40.998609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998639    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998680    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5ddf8fa-c589-4ef9-8299-83ac15845b10-serviceca\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998703    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd239462-09eb-4071-83f6-171abe1cd346-tmp-dir\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998728    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-wtmp\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998748    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23c7a152-2247-4d92-9a1e-8f6a85b4954e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998745    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998790    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998809    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-device-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998822    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd239462-09eb-4071-83f6-171abe1cd346-tmp-dir\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998826    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998872    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-sys\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998904    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23c7a152-2247-4d92-9a1e-8f6a85b4954e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998905    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998934    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efb380a-6530-4db0-88ec-2d59891344f1-tmp-dir\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998960    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998975    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea62f92-610f-4211-9153-474cd90cadaa-metrics-certs\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998978    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efb380a-6530-4db0-88ec-2d59891344f1-tmp-dir\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999001    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-device-dir\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.998988    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999095    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-kubernetes\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999157    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999163    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999163    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999189    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab48927b-5485-4685-84d1-6141d4de8ceb-data-volume\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999206    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-tls\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999223    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-bin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999238    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-daemon-config\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999259    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-kubelet-config\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999307    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab48927b-5485-4685-84d1-6141d4de8ceb-data-volume\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999297    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-tuned\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999361    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999364    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efb380a-6530-4db0-88ec-2d59891344f1-hosts-file\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999384    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-tuned\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999407    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999441    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-daemon-config\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999433    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999408    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ce05b8e-185a-48d6-b18b-5858b2959885-original-pull-secret\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999476    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-tls\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999474    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999519    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999532    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsn4c\" (UniqueName: \"kubernetes.io/projected/23c7a152-2247-4d92-9a1e-8f6a85b4954e-kube-api-access-vsn4c\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999479    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999560    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999587    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999612    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999636    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999667    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999677    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.999688    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999696    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.999727    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.999689    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999727    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-var-lib-kubelet\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.999732    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.499718831 +0000 UTC m=+1.699291746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999782    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.999801    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.499787842 +0000 UTC m=+1.699360753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999796    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdc9936-09aa-4583-95e7-499428c2b9b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-2ldqn\" (UID: \"8bdc9936-09aa-4583-95e7-499428c2b9b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2ldqn"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:40.999818    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.499808549 +0000 UTC m=+1.699381461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999869    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23c7a152-2247-4d92-9a1e-8f6a85b4954e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999836    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8gkt\" (UniqueName: \"kubernetes.io/projected/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-kube-api-access-t8gkt\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999901    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc8vv\" (UniqueName: \"kubernetes.io/projected/b5ddf8fa-c589-4ef9-8299-83ac15845b10-kube-api-access-pc8vv\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999917    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999935    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999950    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999967    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-log-socket\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:40.999982    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000000    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cni-binary-copy\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000015    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-conf-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000034    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000113    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000270    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000286    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000379    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cni-binary-copy\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000434    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.000910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.000542    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.006877 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.006857    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrsp\" (UniqueName: \"kubernetes.io/projected/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-kube-api-access-tgrsp\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.006945 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.006906    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:41.006945 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.006925    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:41.006945 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.006940    6389 projected.go:196] Error preparing data for projected volume kube-api-access-z2fzg for pod openshift-console/console-7ddc794486-fvllw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.007031 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.006959    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpmf\" (UniqueName: \"kubernetes.io/projected/81a971d1-9ef7-43cd-9f79-72ff95212079-kube-api-access-7kpmf\") pod \"ingress-canary-ww6bn\" (UID: \"81a971d1-9ef7-43cd-9f79-72ff95212079\") " pod="openshift-ingress-canary/ingress-canary-ww6bn"
Oct 30 20:04:41.007031 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.007007    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.506992189 +0000 UTC m=+1.706565116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z2fzg" (UniqueName: "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.007099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.007030    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95dh\" (UniqueName: \"kubernetes.io/projected/676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e-kube-api-access-w95dh\") pod \"kube-state-metrics-5dc8d7d89b-qcd5w\" (UID: \"676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qcd5w"
Oct 30 20:04:41.007718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.007698    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bprnz\" (UniqueName: \"kubernetes.io/projected/ab48927b-5485-4685-84d1-6141d4de8ceb-kube-api-access-bprnz\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:41.007937 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.007921    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzlpm\" (UniqueName: \"kubernetes.io/projected/cdaa2197-530d-48e5-899c-5ccc58f4016a-kube-api-access-zzlpm\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.008153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.008135    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q59hq\" (UniqueName: \"kubernetes.io/projected/4ea62f92-610f-4211-9153-474cd90cadaa-kube-api-access-q59hq\") pod \"network-metrics-daemon-c9kfm\" (UID: \"4ea62f92-610f-4211-9153-474cd90cadaa\") " pod="openshift-multus/network-metrics-daemon-c9kfm"
Oct 30 20:04:41.009172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.008378    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk2l2\" (UniqueName: \"kubernetes.io/projected/5efb380a-6530-4db0-88ec-2d59891344f1-kube-api-access-dk2l2\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:41.009172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.008658    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftc9t\" (UniqueName: \"kubernetes.io/projected/26dcecb1-c6af-49e9-a840-b7197582b0cb-kube-api-access-ftc9t\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.012016 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.011998    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:41.012016 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.012018    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:41.012221 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.012032    6389 projected.go:196] Error preparing data for projected volume kube-api-access-5fzc7 for pod openshift-console/console-69d97cbb97-zgnfn: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.012221 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.012067    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7 podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:41.512056929 +0000 UTC m=+1.711629865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5fzc7" (UniqueName: "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.012543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.012522    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tr6l\" (UniqueName: \"kubernetes.io/projected/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-kube-api-access-7tr6l\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.012586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.012571    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxq75\" (UniqueName: \"kubernetes.io/projected/dd239462-09eb-4071-83f6-171abe1cd346-kube-api-access-wxq75\") pod \"dns-default-t7h8t\" (UID: \"dd239462-09eb-4071-83f6-171abe1cd346\") " pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:41.012735 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.012720    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9fnl\" (UniqueName: \"kubernetes.io/projected/a8d6d749-b397-476f-83ca-774cc1aa5a0d-kube-api-access-p9fnl\") pod \"monitoring-plugin-77f4f76594-4tl7l\" (UID: \"a8d6d749-b397-476f-83ca-774cc1aa5a0d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:41.015403 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.015383    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsn4c\" (UniqueName: \"kubernetes.io/projected/23c7a152-2247-4d92-9a1e-8f6a85b4954e-kube-api-access-vsn4c\") pod \"openshift-state-metrics-7bd7489957-rlcdw\" (UID: \"23c7a152-2247-4d92-9a1e-8f6a85b4954e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rlcdw"
Oct 30 20:04:41.015470 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.015450    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") pod \"alertmanager-main-0\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:04:41.016457 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.016428    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8gkt\" (UniqueName: \"kubernetes.io/projected/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-kube-api-access-t8gkt\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.016606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.016573    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc8vv\" (UniqueName: \"kubernetes.io/projected/b5ddf8fa-c589-4ef9-8299-83ac15845b10-kube-api-access-pc8vv\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:41.100914 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.100872    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-modprobe-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.100914 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.100918    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-kubelet\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.100968    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-kubelet\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101009    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-run\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101026    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-bin\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101042    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cnibin\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101041    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-modprobe-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101057    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101080    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-bin\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101079    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-host\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101109    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-cnibin\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101112    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-host\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101158    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-multus\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101112    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-multus\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101164    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-run\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101188    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101193    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-hostroot\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101223    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-hostroot\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101234    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101257    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-var-lib-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101275    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysconfig\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101292    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-conf\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101317    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-dbus\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101334    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-var-lib-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101356    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-systemd\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101366    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysconfig\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101379    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-kubelet\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101370    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-d\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101401    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-dbus\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101416    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101438    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-systemd\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101447    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-sysctl-conf\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101465    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101472    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101486    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-lib-modules\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101511    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-multus-certs\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101542    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ba6f137-f70b-41a4-9e2b-871bb10b4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-49thc\" (UID: \"6ba6f137-f70b-41a4-9e2b-871bb10b4f07\") " pod="openshift-multus/multus-additional-cni-plugins-49thc"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101541    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5ddf8fa-c589-4ef9-8299-83ac15845b10-host\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101571    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5ddf8fa-c589-4ef9-8299-83ac15845b10-host\") pod \"node-ca-zhts8\" (UID: \"b5ddf8fa-c589-4ef9-8299-83ac15845b10\") " pod="openshift-image-registry/node-ca-zhts8"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101418    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-kubelet\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101567    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-multus-certs\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101591    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-systemd-units\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101547    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-lib-modules\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.101605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101620    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-systemd-units\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101628    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-socket-dir-parent\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101651    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-etc-kubernetes\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101665    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-netns\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101680    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-etc-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101695    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-ovn\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101713    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-netns\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101723    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-etc-kubernetes\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101718    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-netd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101734    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-etc-openvswitch\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101755    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-socket-dir-parent\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101762    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab48927b-5485-4685-84d1-6141d4de8ceb-crio-socket\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101758    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-cni-netd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101791    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-sys\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101758    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-ovn\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101817    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-system-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101818    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab48927b-5485-4685-84d1-6141d4de8ceb-crio-socket\") pod \"insights-runtime-extractor-nnnfl\" (UID: \"ab48927b-5485-4685-84d1-6141d4de8ceb\") " pod="openshift-insights/insights-runtime-extractor-nnnfl"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101831    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-sys\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101843    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-netns\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101856    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-system-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101869    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-node-log\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101902    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-netns\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101947    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-root\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101952    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-node-log\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101962    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101977    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cnibin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102003    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-wtmp\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102021    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-sys\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.101981    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-root\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102038    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-kubernetes\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102066    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-etc-kubernetes\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102055    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-cnibin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102085    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-cni-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102098    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-bin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102106    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-sys\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102143    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26dcecb1-c6af-49e9-a840-b7197582b0cb-node-exporter-wtmp\") pod \"node-exporter-d6vx6\" (UID: \"26dcecb1-c6af-49e9-a840-b7197582b0cb\") " pod="openshift-monitoring/node-exporter-d6vx6"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102142    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-kubelet-config\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102168    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efb380a-6530-4db0-88ec-2d59891344f1-hosts-file\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102173    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ce05b8e-185a-48d6-b18b-5858b2959885-kubelet-config\") pod \"global-pull-secret-syncer-sgpdx\" (UID: \"3ce05b8e-185a-48d6-b18b-5858b2959885\") " pod="kube-system/global-pull-secret-syncer-sgpdx"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102185    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-var-lib-cni-bin\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102207    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efb380a-6530-4db0-88ec-2d59891344f1-hosts-file\") pod \"node-resolver-mzzv9\" (UID: \"5efb380a-6530-4db0-88ec-2d59891344f1\") " pod="openshift-dns/node-resolver-mzzv9"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102225    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-var-lib-kubelet\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102248    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-log-socket\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102264    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-conf-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102281    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102285    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ce54b1-7691-41ff-9a3e-56b99c04ced4-var-lib-kubelet\") pod \"tuned-fbmvs\" (UID: \"f4ce54b1-7691-41ff-9a3e-56b99c04ced4\") " pod="openshift-cluster-node-tuning-operator/tuned-fbmvs"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102295    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-log-socket\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102306    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-k8s-cni-cncf-io\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102311    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-multus-conf-dir\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102333    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de1609a-ab4c-448c-bc7f-fcdbcc70ff41-host-run-k8s-cni-cncf-io\") pod \"multus-8lgs5\" (UID: \"3de1609a-ab4c-448c-bc7f-fcdbcc70ff41\") " pod="openshift-multus/multus-8lgs5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102342    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102372    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-slash\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102396    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-systemd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102422    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102470    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-slash\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102485    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-run-systemd\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.102701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.102487    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdaa2197-530d-48e5-899c-5ccc58f4016a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnfz5\" (UID: \"cdaa2197-530d-48e5-899c-5ccc58f4016a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:41.505178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505137    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.505178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505183    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505200    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505221    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505270    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505228    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505298    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505282998 +0000 UTC m=+2.704855931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505306    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505305    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505343    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505330412 +0000 UTC m=+2.704903333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505381    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.505406 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505403    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505393256 +0000 UTC m=+2.704966163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505420    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505414972 +0000 UTC m=+2.704987878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505443    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505445    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505467    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505499    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.50548448 +0000 UTC m=+2.705057401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505504    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505547    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505536916 +0000 UTC m=+2.705109828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505551    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505566    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505601    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505610    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505596244 +0000 UTC m=+2.705169154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505630    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505639    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505656    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505646005 +0000 UTC m=+2.705218961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505674    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505663859 +0000 UTC m=+2.705236815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505693    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505726    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.505762    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505781    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505819    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505821    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505809553 +0000 UTC m=+2.705382463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505845    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505849    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.5058419 +0000 UTC m=+2.705414806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:41.505917 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.505879    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.505868991 +0000 UTC m=+2.705441930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:41.606654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.606613    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:41.606806 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.606733    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:41.606806 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606773    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:41.606806 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606793    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:41.606806 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606803    6389 projected.go:196] Error preparing data for projected volume kube-api-access-5fzc7 for pod openshift-console/console-69d97cbb97-zgnfn: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.606968 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606850    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7 podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.606837815 +0000 UTC m=+2.806410721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5fzc7" (UniqueName: "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.606968 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606876    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:41.606968 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606892    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:41.606968 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606903    6389 projected.go:196] Error preparing data for projected volume kube-api-access-z2fzg for pod openshift-console/console-7ddc794486-fvllw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.606968 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:41.606946    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:42.60693353 +0000 UTC m=+2.806506437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2fzg" (UniqueName: "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:41.831717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:41.831640    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t7h8t"
Oct 30 20:04:42.516682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516644    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.516682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516690    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516722    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516750    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516783    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516793    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516873    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.516895    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516830    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516925    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.516912231 +0000 UTC m=+4.716485140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516928    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516944    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516973    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.516958231 +0000 UTC m=+4.716531167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516837    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516856    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517018    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517012271 +0000 UTC m=+4.716585177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.516790    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517035    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517028223 +0000 UTC m=+4.716601129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517046    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517041523 +0000 UTC m=+4.716614429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517070    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517059306 +0000 UTC m=+4.716632214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517085    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517078958 +0000 UTC m=+4.716651864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.517106    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.517136    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.517153    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.517184    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517189    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.517203    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517226    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517212746 +0000 UTC m=+4.716785666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517250    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517263    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:42.517266 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517271    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:42.518060 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517277    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517269831 +0000 UTC m=+4.716842741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:42.518060 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517301    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:42.518060 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517306    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517299429 +0000 UTC m=+4.716872334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:42.518060 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517316    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517311141 +0000 UTC m=+4.716884047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:42.518060 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.517325    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.517320628 +0000 UTC m=+4.716893533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:42.618521 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.618489    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:42.618668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:42.618615    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:42.618668 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618644    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:42.618668 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618665    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:42.618767 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618676    6389 projected.go:196] Error preparing data for projected volume kube-api-access-z2fzg for pod openshift-console/console-7ddc794486-fvllw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:42.618767 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618717    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.618705753 +0000 UTC m=+4.818278689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2fzg" (UniqueName: "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:42.618767 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618747    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:42.618767 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618766    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:42.618887 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618777    6389 projected.go:196] Error preparing data for projected volume kube-api-access-5fzc7 for pod openshift-console/console-69d97cbb97-zgnfn: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:42.618887 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:42.618821    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7 podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:44.618808599 +0000 UTC m=+4.818381519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5fzc7" (UniqueName: "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:43.037110 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:43.037079    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:43.899074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:43.899041    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:04:44.398781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.398754    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:44.398996 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.398982    6389 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 20:04:44.540363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540332    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.540363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540366    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540384    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540422    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540449    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540473    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540503    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540533    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540540    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540523481 +0000 UTC m=+8.740096387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540565    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540549333 +0000 UTC m=+8.740122239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:44.540584 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540583    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540574916 +0000 UTC m=+8.740147823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540506    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540605    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540616    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540648    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540637228 +0000 UTC m=+8.740210157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540668    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540684    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540674783 +0000 UTC m=+8.740247695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540667    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540700    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540691608 +0000 UTC m=+8.740264517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540704    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540742    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540729894 +0000 UTC m=+8.740302815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540788    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540813    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540860    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540885    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540876212 +0000 UTC m=+8.740449138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:44.540911 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540886    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.540927    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.540916082 +0000 UTC m=+8.740488991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540944    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540975    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.540992    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.541044    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.541061    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.541063    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.541089    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.541074381 +0000 UTC m=+8.740647297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.541114    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.541097614 +0000 UTC m=+8.740670546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:44.541408 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.541164    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.541152032 +0000 UTC m=+8.740724937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:44.642043 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.642010    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:44.642102    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642161    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642181    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642191    6389 projected.go:196] Error preparing data for projected volume kube-api-access-z2fzg for pod openshift-console/console-7ddc794486-fvllw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642196    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642206    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:44.642218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642216    6389 projected.go:196] Error preparing data for projected volume kube-api-access-5fzc7 for pod openshift-console/console-69d97cbb97-zgnfn: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:44.642452 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642233    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.642221979 +0000 UTC m=+8.841794913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2fzg" (UniqueName: "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:44.642452 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:44.642248    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7 podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:48.642238728 +0000 UTC m=+8.841811674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5fzc7" (UniqueName: "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:48.587152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587076    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.587152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587150    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587186    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587225    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587232    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587238    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587261    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587295    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587277934 +0000 UTC m=+16.786850858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587322    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587324    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587363    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587380    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587361947 +0000 UTC m=+16.786934860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587333    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587395    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587354    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587394    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587404    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587394416 +0000 UTC m=+16.786967323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587564    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587545594 +0000 UTC m=+16.787118515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587583    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587573548 +0000 UTC m=+16.787146457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587598    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587591251 +0000 UTC m=+16.787164165 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:48.587655 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587625    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587616227 +0000 UTC m=+16.787189136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587693    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587727    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587780    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587797    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587802    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587824    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587815529 +0000 UTC m=+16.787388440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587852    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587842142 +0000 UTC m=+16.787415048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587854    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587807    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.587883    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587895    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587883259 +0000 UTC m=+16.787456168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587855    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587921    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587949    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587936889 +0000 UTC m=+16.787509818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:48.588382 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.587963    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.587957957 +0000 UTC m=+16.787530863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:48.688547 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.688511    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:48.688727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:48.688609    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:48.688727 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688652    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:48.688727 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688671    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:48.688727 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688681    6389 projected.go:196] Error preparing data for projected volume kube-api-access-z2fzg for pod openshift-console/console-7ddc794486-fvllw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:48.688943 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688732    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:48.688943 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688749    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:48.688943 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688758    6389 projected.go:196] Error preparing data for projected volume kube-api-access-5fzc7 for pod openshift-console/console-69d97cbb97-zgnfn: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:48.688943 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688737    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.68872214 +0000 UTC m=+16.888295050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2fzg" (UniqueName: "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:48.688943 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:48.688810    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7 podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:04:56.688795551 +0000 UTC m=+16.888368459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5fzc7" (UniqueName: "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:49.391399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:49.391365    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4tl7l"
Oct 30 20:04:49.785632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:49.785606    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnfz5"
Oct 30 20:04:49.826504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:49.826467    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:49.925542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:49.925475    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:55.789609 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:55.789522    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:04:56.665555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665522    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.665555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665557    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665586    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665648    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665663    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665687    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665666    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665721    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.665702822 +0000 UTC m=+32.865275748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:56.665731 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665729    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665752    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.665742644 +0000 UTC m=+32.865315562 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665769    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.665761386 +0000 UTC m=+32.865334292 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665786    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665831    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665853    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.66584182 +0000 UTC m=+32.865414727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665871    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665890    6389 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665899    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665940    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665961    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.665943144 +0000 UTC m=+32.865516062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.665947    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.665970 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665896    6389 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665993    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666002    6389 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.665981    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.665970593 +0000 UTC m=+32.865543500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666043    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.666029306 +0000 UTC m=+32.865602215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666057    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.666051003 +0000 UTC m=+32.865623909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666071    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.66606419 +0000 UTC m=+32.865637096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-config" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.666099    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.666135    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.666159    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666195    6389 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666212    6389 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666221    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.66621331 +0000 UTC m=+32.865786220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666214    6389 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666252    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.666241085 +0000 UTC m=+32.865814010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"service-ca" not registered
Oct 30 20:04:56.666464 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.666341    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.666318984 +0000 UTC m=+32.865891893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 20:04:56.766945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.766918    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"console-7ddc794486-fvllw\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") " pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:04:56.767091 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767043    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:56.767091 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767058    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:56.767091 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767068    6389 projected.go:196] Error preparing data for projected volume kube-api-access-z2fzg for pod openshift-console/console-7ddc794486-fvllw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:56.767279 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767098    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg podName:ac7a756f-0197-4cfb-bb53-dfde71285608 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.767087745 +0000 UTC m=+32.966660650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2fzg" (UniqueName: "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg") pod "console-7ddc794486-fvllw" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:56.767279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:04:56.767043    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"console-69d97cbb97-zgnfn\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") " pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:04:56.767279 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767138    6389 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 20:04:56.767279 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767160    6389 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 20:04:56.767279 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767172    6389 projected.go:196] Error preparing data for projected volume kube-api-access-5fzc7 for pod openshift-console/console-69d97cbb97-zgnfn: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:04:56.767279 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:04:56.767220    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7 podName:2a590085-c2de-41b6-aaff-a65de1a3a1c9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:12.767206854 +0000 UTC m=+32.966779762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5fzc7" (UniqueName: "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7") pod "console-69d97cbb97-zgnfn" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 20:05:02.890629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:02.890591    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9jh8v"
Oct 30 20:05:04.540750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:04.540715    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:04.647110 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:04.647076    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:05.907845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:05.907796    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-69d97cbb97-zgnfn" podUID="2a590085-c2de-41b6-aaff-a65de1a3a1c9" containerName="console" containerID="cri-o://b1b6f01507a5cf7e55d0c795a7d03f41052bdd438ad9e271ba5871dffb161a1b" gracePeriod=15
Oct 30 20:05:05.910188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:05.910130    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7ddc794486-fvllw" podUID="ac7a756f-0197-4cfb-bb53-dfde71285608" containerName="console" containerID="cri-o://cbd97c9332b35f673d94331356ca8862c73fe8f6f73c1502d47b3ca1723dff39" gracePeriod=15
Oct 30 20:05:06.062791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.062757    6389 generic.go:356] "Generic (PLEG): container finished" podID="ac7a756f-0197-4cfb-bb53-dfde71285608" containerID="cbd97c9332b35f673d94331356ca8862c73fe8f6f73c1502d47b3ca1723dff39" exitCode=0
Oct 30 20:05:06.062971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.062835    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddc794486-fvllw" event={"ID":"ac7a756f-0197-4cfb-bb53-dfde71285608","Type":"ContainerDied","Data":"cbd97c9332b35f673d94331356ca8862c73fe8f6f73c1502d47b3ca1723dff39"}
Oct 30 20:05:06.064691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.064671    6389 generic.go:356] "Generic (PLEG): container finished" podID="2a590085-c2de-41b6-aaff-a65de1a3a1c9" containerID="b1b6f01507a5cf7e55d0c795a7d03f41052bdd438ad9e271ba5871dffb161a1b" exitCode=0
Oct 30 20:05:06.064870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.064823    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69d97cbb97-zgnfn" event={"ID":"2a590085-c2de-41b6-aaff-a65de1a3a1c9","Type":"ContainerDied","Data":"b1b6f01507a5cf7e55d0c795a7d03f41052bdd438ad9e271ba5871dffb161a1b"}
Oct 30 20:05:06.174248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.174225    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:05:06.176810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.176793    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:05:06.262731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262704    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.262897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262748    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.262897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262776    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.262897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262802    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.262897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262829    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.262897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262858    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.262897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262889    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262919    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262954    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.262989    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263035    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263071    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263102    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263161    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.263217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263198    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") pod \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\" (UID: \"2a590085-c2de-41b6-aaff-a65de1a3a1c9\") "
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263248    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") pod \"ac7a756f-0197-4cfb-bb53-dfde71285608\" (UID: \"ac7a756f-0197-4cfb-bb53-dfde71285608\") "
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263261    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263283    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263319    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263348    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263530    6389 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-console-config\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263543    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263551    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263550    6389 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-service-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263577    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.263593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263594    6389 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-config\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.264133 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263611    6389 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-oauth-serving-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.264133 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263809    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:05:06.264133 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.263811    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.264294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.264162    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:06.265786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.265761    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:06.265891 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.265851    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "kube-api-access-z2fzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:05:06.265983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.265963    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:06.266041 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.265980    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:06.266140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.266109    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert" pod "ac7a756f-0197-4cfb-bb53-dfde71285608" (UID: "ac7a756f-0197-4cfb-bb53-dfde71285608"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:06.266190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.266162    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7" pod "2a590085-c2de-41b6-aaff-a65de1a3a1c9" (UID: "2a590085-c2de-41b6-aaff-a65de1a3a1c9"). InnerVolumeSpecName "kube-api-access-5fzc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:05:06.364649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364626    6389 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-oauth-config\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364646    6389 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-trusted-ca-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364654    6389 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-serving-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364664    6389 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a590085-c2de-41b6-aaff-a65de1a3a1c9-tmp\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364672    6389 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a756f-0197-4cfb-bb53-dfde71285608-tmp\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364679    6389 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a590085-c2de-41b6-aaff-a65de1a3a1c9-console-oauth-config\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364687    6389 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-oauth-serving-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364695    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fzc7\" (UniqueName: \"kubernetes.io/projected/2a590085-c2de-41b6-aaff-a65de1a3a1c9-kube-api-access-5fzc7\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364705    6389 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac7a756f-0197-4cfb-bb53-dfde71285608-trusted-ca-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364714    6389 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a590085-c2de-41b6-aaff-a65de1a3a1c9-service-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364723    6389 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac7a756f-0197-4cfb-bb53-dfde71285608-console-serving-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:06.364776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:06.364731    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2fzg\" (UniqueName: \"kubernetes.io/projected/ac7a756f-0197-4cfb-bb53-dfde71285608-kube-api-access-z2fzg\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:07.069770 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.069741    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69d97cbb97-zgnfn"
Oct 30 20:05:07.069770 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.069747    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69d97cbb97-zgnfn" event={"ID":"2a590085-c2de-41b6-aaff-a65de1a3a1c9","Type":"ContainerDied","Data":"e19d5cf70711431fb6cedb348c79c263b02e09df10b098894f517f8e035f97bd"}
Oct 30 20:05:07.070310 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.069807    6389 scope.go:122] "RemoveContainer" containerID="b1b6f01507a5cf7e55d0c795a7d03f41052bdd438ad9e271ba5871dffb161a1b"
Oct 30 20:05:07.071288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.071165    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddc794486-fvllw"
Oct 30 20:05:07.071288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.071164    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddc794486-fvllw" event={"ID":"ac7a756f-0197-4cfb-bb53-dfde71285608","Type":"ContainerDied","Data":"f3e7e2d4e1e5d1480c23885c7b0bbcbc050f32ae1153cdd6197286562048ed9b"}
Oct 30 20:05:07.078746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.078683    6389 scope.go:122] "RemoveContainer" containerID="cbd97c9332b35f673d94331356ca8862c73fe8f6f73c1502d47b3ca1723dff39"
Oct 30 20:05:07.094510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.094473    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69d97cbb97-zgnfn"]
Oct 30 20:05:07.099655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.099638    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69d97cbb97-zgnfn"]
Oct 30 20:05:07.112163 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.112140    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddc794486-fvllw"]
Oct 30 20:05:07.117228 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.117209    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ddc794486-fvllw"]
Oct 30 20:05:07.914536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.914502    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a590085-c2de-41b6-aaff-a65de1a3a1c9" path="/var/lib/kubelet/pods/2a590085-c2de-41b6-aaff-a65de1a3a1c9/volumes"
Oct 30 20:05:07.914897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:07.914884    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac7a756f-0197-4cfb-bb53-dfde71285608" path="/var/lib/kubelet/pods/ac7a756f-0197-4cfb-bb53-dfde71285608/volumes"
Oct 30 20:05:13.901848 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:13.901818    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zv26m"
Oct 30 20:05:25.790526 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:25.790486    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:05:29.802850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:29.802816    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:29.912168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:29.912132    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:32.694799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:32.694767    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:36.256707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:36.256675    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:36.357175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:36.357149    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:40.606090 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606051    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:05:40.606605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606418    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="alertmanager" containerID="cri-o://06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262" gracePeriod=120
Oct 30 20:05:40.606605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606515    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="prom-label-proxy" containerID="cri-o://8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610" gracePeriod=120
Oct 30 20:05:40.606605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606549    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy" containerID="cri-o://7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3" gracePeriod=120
Oct 30 20:05:40.606605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606559    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="config-reloader" containerID="cri-o://9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d" gracePeriod=120
Oct 30 20:05:40.606822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606500    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-metric" containerID="cri-o://2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc" gracePeriod=120
Oct 30 20:05:40.606822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.606561    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-web" containerID="cri-o://d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9" gracePeriod=120
Oct 30 20:05:40.811576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:40.811540    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:41.186367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186331    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610" exitCode=0
Oct 30 20:05:41.186367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186356    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3" exitCode=0
Oct 30 20:05:41.186367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186365    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9" exitCode=0
Oct 30 20:05:41.186367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186370    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d" exitCode=0
Oct 30 20:05:41.186660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186379    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262" exitCode=0
Oct 30 20:05:41.186660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186405    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610"}
Oct 30 20:05:41.186660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186490    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3"}
Oct 30 20:05:41.186660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186517    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9"}
Oct 30 20:05:41.186660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186541    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d"}
Oct 30 20:05:41.186660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.186564    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262"}
Oct 30 20:05:41.793381 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793353    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-846957c67-krjg8"]
Oct 30 20:05:41.793756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793642    6389 cpu_manager.go:414] "containerMap: removing container" podUID="2a590085-c2de-41b6-aaff-a65de1a3a1c9" containerName="console"
Oct 30 20:05:41.793756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793653    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a590085-c2de-41b6-aaff-a65de1a3a1c9" containerName="console"
Oct 30 20:05:41.793756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793670    6389 cpu_manager.go:414] "containerMap: removing container" podUID="ac7a756f-0197-4cfb-bb53-dfde71285608" containerName="console"
Oct 30 20:05:41.793756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793675    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac7a756f-0197-4cfb-bb53-dfde71285608" containerName="console"
Oct 30 20:05:41.793756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793726    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a590085-c2de-41b6-aaff-a65de1a3a1c9" containerName="console"
Oct 30 20:05:41.793756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.793735    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac7a756f-0197-4cfb-bb53-dfde71285608" containerName="console"
Oct 30 20:05:41.795727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.795698    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.800022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.799984    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 20:05:41.800356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.800333    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 20:05:41.800466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.800361    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 20:05:41.800466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.800391    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fckx2\""
Oct 30 20:05:41.800466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.800436    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 20:05:41.803508 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.803490    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 20:05:41.803616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.803525    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 20:05:41.808097 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.808076    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846957c67-krjg8"]
Oct 30 20:05:41.842078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.842060    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:41.881460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.881431    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:41.904203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.904134    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 20:05:41.904298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.904202    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 20:05:41.913714 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.913688    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-config\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.913833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.913736    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-serving-cert\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.913833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.913814    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-trusted-ca-bundle\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.913939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.913847    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-service-ca\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.913939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.913898    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-oauth-config\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.914046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.913954    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jslp2\" (UniqueName: \"kubernetes.io/projected/4c0f669d-17c7-4226-afe0-3593bc17a4da-kube-api-access-jslp2\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.914046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.914031    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c0f669d-17c7-4226-afe0-3593bc17a4da-tmp\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.914181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.914076    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-oauth-serving-cert\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:41.984626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:41.984597    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:42.015059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015034    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015065    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015096    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015138    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015160    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015186    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015217    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015248    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015282    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015312    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015352    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015401    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015467    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") pod \"51cefd7f-e086-40ff-b71b-c796ea9489e2\" (UID: \"51cefd7f-e086-40ff-b71b-c796ea9489e2\") "
Oct 30 20:05:42.015802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015515    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:42.015802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015589    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-oauth-config\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.015802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015625    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jslp2\" (UniqueName: \"kubernetes.io/projected/4c0f669d-17c7-4226-afe0-3593bc17a4da-kube-api-access-jslp2\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.015802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015656    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c0f669d-17c7-4226-afe0-3593bc17a4da-tmp\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.015802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015712    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-oauth-serving-cert\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.015802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015763    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-config\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.016077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015805    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-serving-cert\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.016077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015838    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-trusted-ca-bundle\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.016077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015865    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-service-ca\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.016077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.015931    6389 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.016729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.016517    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:42.016729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.016631    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:05:42.017017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.016993    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-service-ca\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.017380 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.017353    6389 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 20:05:42.019384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.018484    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c0f669d-17c7-4226-afe0-3593bc17a4da-tmp\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.019384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.018765    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.019384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.019026    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:05:42.019384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.019195    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.019384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.019251    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.019384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.019334    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-trusted-ca-bundle\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.019727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.019498    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-config\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.020901 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.020876    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c0f669d-17c7-4226-afe0-3593bc17a4da-oauth-serving-cert\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.021031 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.021007    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.021092 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.021074    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "kube-api-access-xtshj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:05:42.021174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.020891    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-oauth-config\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.021174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.021086    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.021286 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.021235    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:05:42.021856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.021836    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c0f669d-17c7-4226-afe0-3593bc17a4da-console-serving-cert\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.023621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.023594    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.030239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.030216    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config" pod "51cefd7f-e086-40ff-b71b-c796ea9489e2" (UID: "51cefd7f-e086-40ff-b71b-c796ea9489e2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:05:42.030785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.030763    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jslp2\" (UniqueName: \"kubernetes.io/projected/4c0f669d-17c7-4226-afe0-3593bc17a4da-kube-api-access-jslp2\") pod \"console-846957c67-krjg8\" (UID: \"4c0f669d-17c7-4226-afe0-3593bc17a4da\") " pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.106845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.106797    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:42.116479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116454    6389 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-tls-assets\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116477    6389 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116487    6389 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51cefd7f-e086-40ff-b71b-c796ea9489e2-metrics-client-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116497    6389 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-main-tls\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116506    6389 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-config-out\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116515    6389 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116524    6389 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51cefd7f-e086-40ff-b71b-c796ea9489e2-alertmanager-main-db\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116532    6389 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116545    6389 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-web-config\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116555    6389 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-cluster-tls-config\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116565    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtshj\" (UniqueName: \"kubernetes.io/projected/51cefd7f-e086-40ff-b71b-c796ea9489e2-kube-api-access-xtshj\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.116630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.116587    6389 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51cefd7f-e086-40ff-b71b-c796ea9489e2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:42.193896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.193868    6389 generic.go:356] "Generic (PLEG): container finished" podID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerID="2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc" exitCode=0
Oct 30 20:05:42.194033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.193930    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc"}
Oct 30 20:05:42.194033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.193999    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51cefd7f-e086-40ff-b71b-c796ea9489e2","Type":"ContainerDied","Data":"55d5d80ed4363d8ff78377eb5335dc794bf4a53852a49705e892847357db04df"}
Oct 30 20:05:42.194033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.193938    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.194033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.194032    6389 scope.go:122] "RemoveContainer" containerID="8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610"
Oct 30 20:05:42.200962 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.200896    6389 scope.go:122] "RemoveContainer" containerID="2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc"
Oct 30 20:05:42.207399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.207382    6389 scope.go:122] "RemoveContainer" containerID="7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3"
Oct 30 20:05:42.213707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.213691    6389 scope.go:122] "RemoveContainer" containerID="d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9"
Oct 30 20:05:42.218840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.218821    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:05:42.222236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.221757    6389 scope.go:122] "RemoveContainer" containerID="9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d"
Oct 30 20:05:42.224468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.224425    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:05:42.229174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.229155    6389 scope.go:122] "RemoveContainer" containerID="06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262"
Oct 30 20:05:42.235593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.235578    6389 scope.go:122] "RemoveContainer" containerID="73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74"
Oct 30 20:05:42.241978 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.241962    6389 scope.go:122] "RemoveContainer" containerID="8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610"
Oct 30 20:05:42.242240 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.242223    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610\": container with ID starting with 8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610 not found: ID does not exist" containerID="8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610"
Oct 30 20:05:42.242290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242249    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610"} err="failed to get container status \"8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610\": rpc error: code = NotFound desc = could not find container \"8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610\": container with ID starting with 8fa69d1c7ac4ce99582bb79b88b7123bf0200ae02686c8074d4f79e4bd2af610 not found: ID does not exist"
Oct 30 20:05:42.242290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242280    6389 scope.go:122] "RemoveContainer" containerID="2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc"
Oct 30 20:05:42.242482 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.242468    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc\": container with ID starting with 2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc not found: ID does not exist" containerID="2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc"
Oct 30 20:05:42.242523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242485    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc"} err="failed to get container status \"2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc\": rpc error: code = NotFound desc = could not find container \"2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc\": container with ID starting with 2d214bb23178878e2f559e990e29c1846b00da5498e4f726942ee340c8c688bc not found: ID does not exist"
Oct 30 20:05:42.242523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242499    6389 scope.go:122] "RemoveContainer" containerID="7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3"
Oct 30 20:05:42.242715 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.242697    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3\": container with ID starting with 7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3 not found: ID does not exist" containerID="7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3"
Oct 30 20:05:42.242765 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242720    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3"} err="failed to get container status \"7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3\": rpc error: code = NotFound desc = could not find container \"7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3\": container with ID starting with 7e3522b825bfcd22ab4f969ff5ec330f50ce6bf0e3cf5305b45562a01d9ceff3 not found: ID does not exist"
Oct 30 20:05:42.242765 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242735    6389 scope.go:122] "RemoveContainer" containerID="d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9"
Oct 30 20:05:42.242941 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.242924    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9\": container with ID starting with d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9 not found: ID does not exist" containerID="d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9"
Oct 30 20:05:42.242980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242943    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9"} err="failed to get container status \"d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9\": rpc error: code = NotFound desc = could not find container \"d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9\": container with ID starting with d7b274bf46e4edc8717214e624e5509eb6fa2380101f46a038ae183d9fee45b9 not found: ID does not exist"
Oct 30 20:05:42.242980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.242954    6389 scope.go:122] "RemoveContainer" containerID="9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d"
Oct 30 20:05:42.243157 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.243138    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d\": container with ID starting with 9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d not found: ID does not exist" containerID="9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d"
Oct 30 20:05:42.243200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.243166    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d"} err="failed to get container status \"9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d\": rpc error: code = NotFound desc = could not find container \"9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d\": container with ID starting with 9e2d5c011410ade20a786a69cd46aa943885aad87d8673d1d74053fb3636242d not found: ID does not exist"
Oct 30 20:05:42.243200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.243184    6389 scope.go:122] "RemoveContainer" containerID="06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262"
Oct 30 20:05:42.243397 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.243381    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262\": container with ID starting with 06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262 not found: ID does not exist" containerID="06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262"
Oct 30 20:05:42.243436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.243400    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262"} err="failed to get container status \"06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262\": rpc error: code = NotFound desc = could not find container \"06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262\": container with ID starting with 06494f8b802f541e009b6c1d749f05a71d2589538bc11c20ab247e2faab37262 not found: ID does not exist"
Oct 30 20:05:42.243436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.243411    6389 scope.go:122] "RemoveContainer" containerID="73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74"
Oct 30 20:05:42.243640 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:42.243626    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74\": container with ID starting with 73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74 not found: ID does not exist" containerID="73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74"
Oct 30 20:05:42.243690 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.243641    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74"} err="failed to get container status \"73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74\": rpc error: code = NotFound desc = could not find container \"73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74\": container with ID starting with 73f923b7887ad417a3164d3670ad48032f930ced39d7f2ea3680769419138c74 not found: ID does not exist"
Oct 30 20:05:42.256551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.256531    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:05:42.257288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257252    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="init-config-reloader"
Oct 30 20:05:42.257379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257285    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="init-config-reloader"
Oct 30 20:05:42.257379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257335    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="prom-label-proxy"
Oct 30 20:05:42.257379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257345    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="prom-label-proxy"
Oct 30 20:05:42.257379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257366    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy"
Oct 30 20:05:42.257379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257373    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257400    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-web"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257407    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-web"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257435    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="config-reloader"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257442    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="config-reloader"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257460    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="alertmanager"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257467    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="alertmanager"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257491    6389 cpu_manager.go:414] "containerMap: removing container" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-metric"
Oct 30 20:05:42.257628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257498    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-metric"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257658    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="prom-label-proxy"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257669    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257680    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-web"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257691    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="config-reloader"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257706    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="alertmanager"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257716    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="kube-rbac-proxy-metric"
Oct 30 20:05:42.257973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.257733    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" containerName="init-config-reloader"
Oct 30 20:05:42.263108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.263086    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.268086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268070    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 20:05:42.268086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268076    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 20:05:42.268086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268077    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 20:05:42.268086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268100    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 20:05:42.268386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268146    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 20:05:42.268386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268165    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 20:05:42.268386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268214    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 20:05:42.268386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268318    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9wn4n\""
Oct 30 20:05:42.268615 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.268599    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 20:05:42.271723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.271702    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 20:05:42.280605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.280585    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:05:42.377941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.377916    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846957c67-krjg8"]
Oct 30 20:05:42.382281 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:05:42.382252    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c0f669d_17c7_4226_afe0_3593bc17a4da.slice/crio-3d05d41aaa01b472c8a52706d74fb44796c7cf2b8f4fd69d580e2c0e1ae4b475 WatchSource:0}: Error finding container 3d05d41aaa01b472c8a52706d74fb44796c7cf2b8f4fd69d580e2c0e1ae4b475: Status 404 returned error can't find the container with id 3d05d41aaa01b472c8a52706d74fb44796c7cf2b8f4fd69d580e2c0e1ae4b475
Oct 30 20:05:42.419793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.419768    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-config-volume\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.419890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.419809    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.419890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.419828    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.419979 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.419921    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8pww\" (UniqueName: \"kubernetes.io/projected/fa1af69f-326f-491c-909a-7a63526052d6-kube-api-access-z8pww\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.419979 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.419964    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa1af69f-326f-491c-909a-7a63526052d6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420054 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.419990    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa1af69f-326f-491c-909a-7a63526052d6-config-out\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420054 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420021    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420057    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420072    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fa1af69f-326f-491c-909a-7a63526052d6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420186    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-web-config\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420226    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420256    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa1af69f-326f-491c-909a-7a63526052d6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.420405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.420282    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa1af69f-326f-491c-909a-7a63526052d6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.460981 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.460942    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:42.520765 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520732    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520765 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520765    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520785    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pww\" (UniqueName: \"kubernetes.io/projected/fa1af69f-326f-491c-909a-7a63526052d6-kube-api-access-z8pww\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520809    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa1af69f-326f-491c-909a-7a63526052d6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520850    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa1af69f-326f-491c-909a-7a63526052d6-config-out\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520875    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520903    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520926    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fa1af69f-326f-491c-909a-7a63526052d6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.520956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520954    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-web-config\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.521387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.520985    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.521387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.521008    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa1af69f-326f-491c-909a-7a63526052d6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.521387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.521029    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa1af69f-326f-491c-909a-7a63526052d6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.521387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.521095    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-config-volume\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.521556 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.521388    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fa1af69f-326f-491c-909a-7a63526052d6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.523616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.522494    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa1af69f-326f-491c-909a-7a63526052d6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.523616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.523537    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa1af69f-326f-491c-909a-7a63526052d6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524159    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524234    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa1af69f-326f-491c-909a-7a63526052d6-config-out\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524246    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524400    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-config-volume\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524401    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524446    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa1af69f-326f-491c-909a-7a63526052d6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.524583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.524533    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.525472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.525449    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.525710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.525696    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa1af69f-326f-491c-909a-7a63526052d6-web-config\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.533576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.533558    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8pww\" (UniqueName: \"kubernetes.io/projected/fa1af69f-326f-491c-909a-7a63526052d6-kube-api-access-z8pww\") pod \"alertmanager-main-0\" (UID: \"fa1af69f-326f-491c-909a-7a63526052d6\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.559889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.559863    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:42.585960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.585942    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 20:05:42.869410 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:42.869379    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 20:05:42.872634 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:05:42.872604    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1af69f_326f_491c_909a_7a63526052d6.slice/crio-86607b68d55767db47c7a6fd48ce0a644d5b935afe08d967d24a243a55ce2d97 WatchSource:0}: Error finding container 86607b68d55767db47c7a6fd48ce0a644d5b935afe08d967d24a243a55ce2d97: Status 404 returned error can't find the container with id 86607b68d55767db47c7a6fd48ce0a644d5b935afe08d967d24a243a55ce2d97
Oct 30 20:05:43.198291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.198199    6389 generic.go:356] "Generic (PLEG): container finished" podID="fa1af69f-326f-491c-909a-7a63526052d6" containerID="c808f8d8d0b58620d62f2797b0f163a9d6735b6b7a7ca14e574b96fe0e821b0d" exitCode=0
Oct 30 20:05:43.198442 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.198297    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerDied","Data":"c808f8d8d0b58620d62f2797b0f163a9d6735b6b7a7ca14e574b96fe0e821b0d"}
Oct 30 20:05:43.198442 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.198371    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"86607b68d55767db47c7a6fd48ce0a644d5b935afe08d967d24a243a55ce2d97"}
Oct 30 20:05:43.199940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.199908    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846957c67-krjg8" event={"ID":"4c0f669d-17c7-4226-afe0-3593bc17a4da","Type":"ContainerStarted","Data":"0bfd27b9ba54e43252b49c297c8dd926a402ef4ef2a099ed06bdf10f0a555c40"}
Oct 30 20:05:43.200044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.199952    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846957c67-krjg8" event={"ID":"4c0f669d-17c7-4226-afe0-3593bc17a4da","Type":"ContainerStarted","Data":"3d05d41aaa01b472c8a52706d74fb44796c7cf2b8f4fd69d580e2c0e1ae4b475"}
Oct 30 20:05:43.254074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.254014    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-846957c67-krjg8" podStartSLOduration=2.254001276 podStartE2EDuration="2.254001276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:43.253675451 +0000 UTC m=+63.453248380" watchObservedRunningTime="2026-10-30 20:05:43.254001276 +0000 UTC m=+63.453574201"
Oct 30 20:05:43.911261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:43.911228    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51cefd7f-e086-40ff-b71b-c796ea9489e2" path="/var/lib/kubelet/pods/51cefd7f-e086-40ff-b71b-c796ea9489e2/volumes"
Oct 30 20:05:44.207272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:44.207186    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"a46ab105a4b9447d84decdb573d32432518b940dfa52ba2dea09e09657a63387"}
Oct 30 20:05:44.207272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:44.207252    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"3e903184f62686b9610ab50d8e281ac4678084d71683de72f0adf8986e2d586f"}
Oct 30 20:05:44.207497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:44.207277    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"48c06b56704465e8e988d19d08fee60f8a2e60303834d7f16da46eea4f2d974a"}
Oct 30 20:05:44.207497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:44.207301    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"de5835cb05d690e435ac84794f2044a86a75af5d22196161c0e73071c40eaaa5"}
Oct 30 20:05:44.207497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:44.207323    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"dc9252e7d2ca0be9f26729fbb5ede046684f2115214bf5808e2622f022f29efc"}
Oct 30 20:05:44.207497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:44.207345    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fa1af69f-326f-491c-909a-7a63526052d6","Type":"ContainerStarted","Data":"2197e5ad64b1d59601feb9765ab36ca45f4ad31a6df2036f006311000266b173"}
Oct 30 20:05:48.162434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:48.162394    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:48.261088 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:48.261057    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:48.977727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:48.977694    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:49.077252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:49.077213    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:05:52.107206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:52.107177    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:52.107692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:52.107441    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:52.112047 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:52.112027    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:52.135019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:52.134971    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.126864231 podStartE2EDuration="10.134959211s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.09498ms" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:44.245267321 +0000 UTC m=+64.444840251" watchObservedRunningTime="2026-10-30 20:05:52.134959211 +0000 UTC m=+72.334532133"
Oct 30 20:05:52.237833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:05:52.237811    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-846957c67-krjg8"
Oct 30 20:05:55.790749 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:05:55.790712    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:06:04.545943 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:06:04.545901    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:06:04.644427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:06:04.644394    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:06:04.744562 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:06:04.744533    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:06:12.797410 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:06:12.797378    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:06:12.896970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:06:12.896939    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:06:25.790346 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:06:25.790312    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:06:32.742633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:06:32.742597    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:06:55.790703 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:06:55.790660    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:07:25.790873 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:07:25.790812    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:07:32.795242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:32.795194    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:07:39.146671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:39.146620    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:07:39.258607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:39.258540    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:07:51.282685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:51.282575    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:07:51.394212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:51.394162    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:07:52.043628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.043588    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"]
Oct 30 20:07:52.047107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.047087    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.052470 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.052448    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 20:07:52.053543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.053369    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 20:07:52.053543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.053466    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-5rkln\""
Oct 30 20:07:52.056697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.056676    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"]
Oct 30 20:07:52.125292 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.125266    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.125422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.125308    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n26nc\" (UniqueName: \"kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.125422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.125387    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.226076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.226051    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.226215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.226093    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n26nc\" (UniqueName: \"kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.226215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.226161    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.226504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.226485    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.226541 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.226502    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.239944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.239918    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n26nc\" (UniqueName: \"kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.362159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.362086    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:07:52.633384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.633309    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"]
Oct 30 20:07:52.636733 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:07:52.636708    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4dcca7_0706_4328_8c09_4302280fdf0a.slice/crio-0a673adac0e7f78c2184f21bf798826152dfb644304254f3ffa31ff036da93e7 WatchSource:0}: Error finding container 0a673adac0e7f78c2184f21bf798826152dfb644304254f3ffa31ff036da93e7: Status 404 returned error can't find the container with id 0a673adac0e7f78c2184f21bf798826152dfb644304254f3ffa31ff036da93e7
Oct 30 20:07:52.638559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:52.638543    6389 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:07:53.630165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:53.630110    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t" event={"ID":"8b4dcca7-0706-4328-8c09-4302280fdf0a","Type":"ContainerStarted","Data":"0a673adac0e7f78c2184f21bf798826152dfb644304254f3ffa31ff036da93e7"}
Oct 30 20:07:55.355794 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:55.355755    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:07:55.794909 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:07:55.794874    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:07:58.649483 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:58.649445    6389 generic.go:356] "Generic (PLEG): container finished" podID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerID="677003c13e6aef23c1bdeb31bb5b9e210c27e2ff9b30b2723016e1bf6bbab7cf" exitCode=0
Oct 30 20:07:58.649872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:07:58.649510    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t" event={"ID":"8b4dcca7-0706-4328-8c09-4302280fdf0a","Type":"ContainerDied","Data":"677003c13e6aef23c1bdeb31bb5b9e210c27e2ff9b30b2723016e1bf6bbab7cf"}
Oct 30 20:08:01.662027 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:01.661994    6389 generic.go:356] "Generic (PLEG): container finished" podID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerID="52dbf924459cbe3af9e1609cbafeae2b94f8878cf353dbd33d327264a111a010" exitCode=0
Oct 30 20:08:01.662540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:01.662043    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t" event={"ID":"8b4dcca7-0706-4328-8c09-4302280fdf0a","Type":"ContainerDied","Data":"52dbf924459cbe3af9e1609cbafeae2b94f8878cf353dbd33d327264a111a010"}
Oct 30 20:08:02.242757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:02.242728    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:02.342580 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:02.342527    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:07.687577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:07.687546    6389 generic.go:356] "Generic (PLEG): container finished" podID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerID="df432e697a72ad9a09e30bae40738afb216d4eee0e76693ad9377f0f1fc7445c" exitCode=0
Oct 30 20:08:07.687966 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:07.687622    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t" event={"ID":"8b4dcca7-0706-4328-8c09-4302280fdf0a","Type":"ContainerDied","Data":"df432e697a72ad9a09e30bae40738afb216d4eee0e76693ad9377f0f1fc7445c"}
Oct 30 20:08:08.822164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.822141    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:08:08.960904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.960828    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle\") pod \"8b4dcca7-0706-4328-8c09-4302280fdf0a\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") "
Oct 30 20:08:08.960904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.960874    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util\") pod \"8b4dcca7-0706-4328-8c09-4302280fdf0a\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") "
Oct 30 20:08:08.961074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.960922    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc\" (UniqueName: \"kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc\") pod \"8b4dcca7-0706-4328-8c09-4302280fdf0a\" (UID: \"8b4dcca7-0706-4328-8c09-4302280fdf0a\") "
Oct 30 20:08:08.961490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.961466    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle" pod "8b4dcca7-0706-4328-8c09-4302280fdf0a" (UID: "8b4dcca7-0706-4328-8c09-4302280fdf0a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:08.963156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.963127    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc" pod "8b4dcca7-0706-4328-8c09-4302280fdf0a" (UID: "8b4dcca7-0706-4328-8c09-4302280fdf0a"). InnerVolumeSpecName "kube-api-access-n26nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:08.965013 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:08.964992    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util" pod "8b4dcca7-0706-4328-8c09-4302280fdf0a" (UID: "8b4dcca7-0706-4328-8c09-4302280fdf0a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:09.062232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:09.062199    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n26nc\" (UniqueName: \"kubernetes.io/projected/8b4dcca7-0706-4328-8c09-4302280fdf0a-kube-api-access-n26nc\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:09.062232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:09.062227    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:09.062232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:09.062236    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b4dcca7-0706-4328-8c09-4302280fdf0a-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:09.695926 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:09.695887    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t" event={"ID":"8b4dcca7-0706-4328-8c09-4302280fdf0a","Type":"ContainerDied","Data":"0a673adac0e7f78c2184f21bf798826152dfb644304254f3ffa31ff036da93e7"}
Oct 30 20:08:09.696096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:09.695944    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4ts9t"
Oct 30 20:08:09.696096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:09.695945    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a673adac0e7f78c2184f21bf798826152dfb644304254f3ffa31ff036da93e7"
Oct 30 20:08:13.105001 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:13.104955    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:13.926583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:13.926551    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:15.091499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091467    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"]
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091738    6389 cpu_manager.go:414] "containerMap: removing container" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="pull"
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091750    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="pull"
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091763    6389 cpu_manager.go:414] "containerMap: removing container" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="util"
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091768    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="util"
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091783    6389 cpu_manager.go:414] "containerMap: removing container" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="extract"
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091788    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="extract"
Oct 30 20:08:15.091871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.091846    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b4dcca7-0706-4328-8c09-4302280fdf0a" containerName="extract"
Oct 30 20:08:15.093738 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.093723    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.113835 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.113803    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 20:08:15.113982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.113836    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-dxz8h\""
Oct 30 20:08:15.113982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.113867    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:08:15.114650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.114631    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"]
Oct 30 20:08:15.207836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.207804    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d129e9a-778c-4001-90ae-5df50926312b-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-xtrdv\" (UID: \"5d129e9a-778c-4001-90ae-5df50926312b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.207982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.207849    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pngrw\" (UniqueName: \"kubernetes.io/projected/5d129e9a-778c-4001-90ae-5df50926312b-kube-api-access-pngrw\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-xtrdv\" (UID: \"5d129e9a-778c-4001-90ae-5df50926312b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.309042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.309002    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d129e9a-778c-4001-90ae-5df50926312b-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-xtrdv\" (UID: \"5d129e9a-778c-4001-90ae-5df50926312b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.309163 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.309055    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pngrw\" (UniqueName: \"kubernetes.io/projected/5d129e9a-778c-4001-90ae-5df50926312b-kube-api-access-pngrw\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-xtrdv\" (UID: \"5d129e9a-778c-4001-90ae-5df50926312b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.309426 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.309407    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d129e9a-778c-4001-90ae-5df50926312b-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-xtrdv\" (UID: \"5d129e9a-778c-4001-90ae-5df50926312b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.325160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.325113    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pngrw\" (UniqueName: \"kubernetes.io/projected/5d129e9a-778c-4001-90ae-5df50926312b-kube-api-access-pngrw\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-xtrdv\" (UID: \"5d129e9a-778c-4001-90ae-5df50926312b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.403650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.403591    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"
Oct 30 20:08:15.680788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.680656    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv"]
Oct 30 20:08:15.683188 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:15.683165    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d129e9a_778c_4001_90ae_5df50926312b.slice/crio-9ca7c44fefb453442ed47d3fccc2f5db2087a23b3a4a3d880f053b75e530327d WatchSource:0}: Error finding container 9ca7c44fefb453442ed47d3fccc2f5db2087a23b3a4a3d880f053b75e530327d: Status 404 returned error can't find the container with id 9ca7c44fefb453442ed47d3fccc2f5db2087a23b3a4a3d880f053b75e530327d
Oct 30 20:08:15.717034 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:15.717003    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv" event={"ID":"5d129e9a-778c-4001-90ae-5df50926312b","Type":"ContainerStarted","Data":"9ca7c44fefb453442ed47d3fccc2f5db2087a23b3a4a3d880f053b75e530327d"}
Oct 30 20:08:16.521482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:16.521450    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:16.622634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:16.622604    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:17.726728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:17.726695    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv" event={"ID":"5d129e9a-778c-4001-90ae-5df50926312b","Type":"ContainerStarted","Data":"0e8b15d0f87a7f28902fd13b46d00b363eb65da9827ec36783c4d5f5226aeb3d"}
Oct 30 20:08:17.752769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:17.752707    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-xtrdv" podStartSLOduration=0.902750078 podStartE2EDuration="2.752690066s" totalImagesPullingTime="1.849939988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:17.751404632 +0000 UTC m=+217.950977561" watchObservedRunningTime="2026-10-30 20:08:17.752690066 +0000 UTC m=+217.952262995"
Oct 30 20:08:18.228838 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:18.228787    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:18.398978 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:18.398947    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:18.519175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:18.519077    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:19.122284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.122245    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"]
Oct 30 20:08:19.124799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.124782    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.136271 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.136253    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 20:08:19.136434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.136419    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"61b1ff68c7255ee5c5329fe19a5d96293a9578-ce91a85d-dockercfg-68cgj\""
Oct 30 20:08:19.136474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.136460    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 20:08:19.138480 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.138461    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"]
Oct 30 20:08:19.239953 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.239919    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.240106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.239979    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w4xj\" (UniqueName: \"kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.240106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.240078    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.340447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.340411    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w4xj\" (UniqueName: \"kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.340447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.340453    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.340708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.340496    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.340960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.340940    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.341037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.340977    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.355777 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.355745    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w4xj\" (UniqueName: \"kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.436452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.436372    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:19.470446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.470414    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"]
Oct 30 20:08:19.474101 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.473733    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.487024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.487001    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 20:08:19.487164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.487144    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 20:08:19.487277 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.487259    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xc2mf\""
Oct 30 20:08:19.501136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.501096    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"]
Oct 30 20:08:19.541765 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.541738    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df0c953c-ae59-46fa-8c05-4362c046e8db-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-nxfsh\" (UID: \"df0c953c-ae59-46fa-8c05-4362c046e8db\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.541918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.541782    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prhz6\" (UniqueName: \"kubernetes.io/projected/df0c953c-ae59-46fa-8c05-4362c046e8db-kube-api-access-prhz6\") pod \"cert-manager-webhook-5dbbbf75cc-nxfsh\" (UID: \"df0c953c-ae59-46fa-8c05-4362c046e8db\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.642521 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.642488    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df0c953c-ae59-46fa-8c05-4362c046e8db-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-nxfsh\" (UID: \"df0c953c-ae59-46fa-8c05-4362c046e8db\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.642680 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.642530    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prhz6\" (UniqueName: \"kubernetes.io/projected/df0c953c-ae59-46fa-8c05-4362c046e8db-kube-api-access-prhz6\") pod \"cert-manager-webhook-5dbbbf75cc-nxfsh\" (UID: \"df0c953c-ae59-46fa-8c05-4362c046e8db\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.656661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.656636    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df0c953c-ae59-46fa-8c05-4362c046e8db-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-nxfsh\" (UID: \"df0c953c-ae59-46fa-8c05-4362c046e8db\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.656783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.656717    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prhz6\" (UniqueName: \"kubernetes.io/projected/df0c953c-ae59-46fa-8c05-4362c046e8db-kube-api-access-prhz6\") pod \"cert-manager-webhook-5dbbbf75cc-nxfsh\" (UID: \"df0c953c-ae59-46fa-8c05-4362c046e8db\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:19.711212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.711149    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"]
Oct 30 20:08:19.714006 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:19.713978    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1e91c1_079d_426e_9e14_10cce75dbd17.slice/crio-bbe5d6a9c7df4b5ab2cc167e200810aabfb0f4cd22d6a33b9e39b9af4dc3f002 WatchSource:0}: Error finding container bbe5d6a9c7df4b5ab2cc167e200810aabfb0f4cd22d6a33b9e39b9af4dc3f002: Status 404 returned error can't find the container with id bbe5d6a9c7df4b5ab2cc167e200810aabfb0f4cd22d6a33b9e39b9af4dc3f002
Oct 30 20:08:19.735506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.735480    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5" event={"ID":"da1e91c1-079d-426e-9e14-10cce75dbd17","Type":"ContainerStarted","Data":"bbe5d6a9c7df4b5ab2cc167e200810aabfb0f4cd22d6a33b9e39b9af4dc3f002"}
Oct 30 20:08:19.794160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:19.794065    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:20.064462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:20.064433    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"]
Oct 30 20:08:20.068695 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:20.068666    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf0c953c_ae59_46fa_8c05_4362c046e8db.slice/crio-dcdbfdfe36e05e2145809aee15f74c75164e1e8943b36f7075741f4e9825e649 WatchSource:0}: Error finding container dcdbfdfe36e05e2145809aee15f74c75164e1e8943b36f7075741f4e9825e649: Status 404 returned error can't find the container with id dcdbfdfe36e05e2145809aee15f74c75164e1e8943b36f7075741f4e9825e649
Oct 30 20:08:20.575171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:20.575134    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:20.740743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:20.740698    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh" event={"ID":"df0c953c-ae59-46fa-8c05-4362c046e8db","Type":"ContainerStarted","Data":"dcdbfdfe36e05e2145809aee15f74c75164e1e8943b36f7075741f4e9825e649"}
Oct 30 20:08:20.742181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:20.742154    6389 generic.go:356] "Generic (PLEG): container finished" podID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerID="511a3ed53817cef448264e6cc5bac9b2e1fdcae3bfdb57e4539a269abcedc200" exitCode=0
Oct 30 20:08:20.742312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:20.742205    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5" event={"ID":"da1e91c1-079d-426e-9e14-10cce75dbd17","Type":"ContainerDied","Data":"511a3ed53817cef448264e6cc5bac9b2e1fdcae3bfdb57e4539a269abcedc200"}
Oct 30 20:08:20.977937 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:20.977909    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:23.315380 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.315338    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:23.415194 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.415162    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:23.756756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.756726    6389 generic.go:356] "Generic (PLEG): container finished" podID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerID="7a0c943969d835e66cb02b27b3a49619840d80a788c0585b3617d91527748534" exitCode=0
Oct 30 20:08:23.756889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.756793    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5" event={"ID":"da1e91c1-079d-426e-9e14-10cce75dbd17","Type":"ContainerDied","Data":"7a0c943969d835e66cb02b27b3a49619840d80a788c0585b3617d91527748534"}
Oct 30 20:08:23.758367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.758296    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh" event={"ID":"df0c953c-ae59-46fa-8c05-4362c046e8db","Type":"ContainerStarted","Data":"61078ce5cfe356c47debcb8976905838578cf56c1c1be5d452127c2ad8892317"}
Oct 30 20:08:23.758449 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.758426    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:23.808841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:23.808652    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh" podStartSLOduration=1.6696076720000002 podStartE2EDuration="4.808310754s" totalImagesPullingTime="3.138703082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:23.807570469 +0000 UTC m=+224.007143400" watchObservedRunningTime="2026-10-30 20:08:23.808310754 +0000 UTC m=+224.007883683"
Oct 30 20:08:24.764842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:24.764804    6389 generic.go:356] "Generic (PLEG): container finished" podID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerID="afd055774bbe7c53e2034170970a29ea5655f3a9486a6e16ea015f6aed0177c9" exitCode=0
Oct 30 20:08:24.765274 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:24.764887    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5" event={"ID":"da1e91c1-079d-426e-9e14-10cce75dbd17","Type":"ContainerDied","Data":"afd055774bbe7c53e2034170970a29ea5655f3a9486a6e16ea015f6aed0177c9"}
Oct 30 20:08:25.794665 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:08:25.794629    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:08:25.888916 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:25.888890    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:26.000104 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.000072    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle\") pod \"da1e91c1-079d-426e-9e14-10cce75dbd17\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") "
Oct 30 20:08:26.000311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.000158    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util\") pod \"da1e91c1-079d-426e-9e14-10cce75dbd17\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") "
Oct 30 20:08:26.000311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.000214    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj\" (UniqueName: \"kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj\") pod \"da1e91c1-079d-426e-9e14-10cce75dbd17\" (UID: \"da1e91c1-079d-426e-9e14-10cce75dbd17\") "
Oct 30 20:08:26.000518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.000489    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle" pod "da1e91c1-079d-426e-9e14-10cce75dbd17" (UID: "da1e91c1-079d-426e-9e14-10cce75dbd17"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.002272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.002242    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj" pod "da1e91c1-079d-426e-9e14-10cce75dbd17" (UID: "da1e91c1-079d-426e-9e14-10cce75dbd17"). InnerVolumeSpecName "kube-api-access-2w4xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:26.003899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.003879    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util" pod "da1e91c1-079d-426e-9e14-10cce75dbd17" (UID: "da1e91c1-079d-426e-9e14-10cce75dbd17"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.101444 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.101364    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.101444 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.101393    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da1e91c1-079d-426e-9e14-10cce75dbd17-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.101444 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.101402    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w4xj\" (UniqueName: \"kubernetes.io/projected/da1e91c1-079d-426e-9e14-10cce75dbd17-kube-api-access-2w4xj\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.647615 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647580    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"]
Oct 30 20:08:26.647851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647839    6389 cpu_manager.go:414] "containerMap: removing container" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="extract"
Oct 30 20:08:26.647895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647852    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="extract"
Oct 30 20:08:26.647895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647862    6389 cpu_manager.go:414] "containerMap: removing container" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="pull"
Oct 30 20:08:26.647895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647866    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="pull"
Oct 30 20:08:26.647895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647874    6389 cpu_manager.go:414] "containerMap: removing container" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="util"
Oct 30 20:08:26.647895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647879    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="util"
Oct 30 20:08:26.648270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.647938    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="da1e91c1-079d-426e-9e14-10cce75dbd17" containerName="extract"
Oct 30 20:08:26.652007 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.651988    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.659667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.659647    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-t6jv6\""
Oct 30 20:08:26.659969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.659945    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"]
Oct 30 20:08:26.778923 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.778898    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5"
Oct 30 20:08:26.779047 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.778890    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc858zlk5" event={"ID":"da1e91c1-079d-426e-9e14-10cce75dbd17","Type":"ContainerDied","Data":"bbe5d6a9c7df4b5ab2cc167e200810aabfb0f4cd22d6a33b9e39b9af4dc3f002"}
Oct 30 20:08:26.779103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.779046    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbe5d6a9c7df4b5ab2cc167e200810aabfb0f4cd22d6a33b9e39b9af4dc3f002"
Oct 30 20:08:26.804743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.804717    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d26g\" (UniqueName: \"kubernetes.io/projected/716f1634-5014-4fe4-aabb-6df24b12bc35-kube-api-access-2d26g\") pod \"cert-manager-cainjector-5c57bbd4fc-4lclf\" (UID: \"716f1634-5014-4fe4-aabb-6df24b12bc35\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.805043 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.804757    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f1634-5014-4fe4-aabb-6df24b12bc35-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4lclf\" (UID: \"716f1634-5014-4fe4-aabb-6df24b12bc35\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.905788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.905723    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f1634-5014-4fe4-aabb-6df24b12bc35-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4lclf\" (UID: \"716f1634-5014-4fe4-aabb-6df24b12bc35\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.905788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.905783    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d26g\" (UniqueName: \"kubernetes.io/projected/716f1634-5014-4fe4-aabb-6df24b12bc35-kube-api-access-2d26g\") pod \"cert-manager-cainjector-5c57bbd4fc-4lclf\" (UID: \"716f1634-5014-4fe4-aabb-6df24b12bc35\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.919205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.919180    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/716f1634-5014-4fe4-aabb-6df24b12bc35-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4lclf\" (UID: \"716f1634-5014-4fe4-aabb-6df24b12bc35\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.919293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.919269    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d26g\" (UniqueName: \"kubernetes.io/projected/716f1634-5014-4fe4-aabb-6df24b12bc35-kube-api-access-2d26g\") pod \"cert-manager-cainjector-5c57bbd4fc-4lclf\" (UID: \"716f1634-5014-4fe4-aabb-6df24b12bc35\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:26.965398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:26.965372    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"
Oct 30 20:08:27.239585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:27.239555    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf"]
Oct 30 20:08:27.242841 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:27.242811    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod716f1634_5014_4fe4_aabb_6df24b12bc35.slice/crio-f2728f52b4604f36ee0ecd529400e2e9d7f5819add48b9d7ccf97930e856f346 WatchSource:0}: Error finding container f2728f52b4604f36ee0ecd529400e2e9d7f5819add48b9d7ccf97930e856f346: Status 404 returned error can't find the container with id f2728f52b4604f36ee0ecd529400e2e9d7f5819add48b9d7ccf97930e856f346
Oct 30 20:08:27.795454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:27.795414    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf" event={"ID":"716f1634-5014-4fe4-aabb-6df24b12bc35","Type":"ContainerStarted","Data":"237a7951c38185c39b7368e43611e6bd53aa59a45a5c400ce09cf6b3528c2be7"}
Oct 30 20:08:27.795666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:27.795472    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf" event={"ID":"716f1634-5014-4fe4-aabb-6df24b12bc35","Type":"ContainerStarted","Data":"f2728f52b4604f36ee0ecd529400e2e9d7f5819add48b9d7ccf97930e856f346"}
Oct 30 20:08:27.816967 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:27.816282    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4lclf" podStartSLOduration=1.816262576 podStartE2EDuration="1.816262576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:27.815382807 +0000 UTC m=+228.014955737" watchObservedRunningTime="2026-10-30 20:08:27.816262576 +0000 UTC m=+228.015835505"
Oct 30 20:08:28.063874 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.063788    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:28.167925 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.167892    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:28.575067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.575036    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:28.808278 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.808249    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-dcfxs"]
Oct 30 20:08:28.811250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.811232    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:28.817439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.817419    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7jtwm\""
Oct 30 20:08:28.821854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.821828    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-dcfxs"]
Oct 30 20:08:28.923033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.922947    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7bfb1811-7f89-4341-bf13-65381da1e379-bound-sa-token\") pod \"cert-manager-597b95cb84-dcfxs\" (UID: \"7bfb1811-7f89-4341-bf13-65381da1e379\") " pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:28.923033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:28.923011    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdbn7\" (UniqueName: \"kubernetes.io/projected/7bfb1811-7f89-4341-bf13-65381da1e379-kube-api-access-pdbn7\") pod \"cert-manager-597b95cb84-dcfxs\" (UID: \"7bfb1811-7f89-4341-bf13-65381da1e379\") " pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:29.024443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.024410    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7bfb1811-7f89-4341-bf13-65381da1e379-bound-sa-token\") pod \"cert-manager-597b95cb84-dcfxs\" (UID: \"7bfb1811-7f89-4341-bf13-65381da1e379\") " pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:29.024648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.024472    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdbn7\" (UniqueName: \"kubernetes.io/projected/7bfb1811-7f89-4341-bf13-65381da1e379-kube-api-access-pdbn7\") pod \"cert-manager-597b95cb84-dcfxs\" (UID: \"7bfb1811-7f89-4341-bf13-65381da1e379\") " pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:29.038305 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.038268    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7bfb1811-7f89-4341-bf13-65381da1e379-bound-sa-token\") pod \"cert-manager-597b95cb84-dcfxs\" (UID: \"7bfb1811-7f89-4341-bf13-65381da1e379\") " pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:29.038432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.038337    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdbn7\" (UniqueName: \"kubernetes.io/projected/7bfb1811-7f89-4341-bf13-65381da1e379-kube-api-access-pdbn7\") pod \"cert-manager-597b95cb84-dcfxs\" (UID: \"7bfb1811-7f89-4341-bf13-65381da1e379\") " pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:29.122976 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.122942    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-dcfxs"
Oct 30 20:08:29.393527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.393494    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-dcfxs"]
Oct 30 20:08:29.396434 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:29.396408    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bfb1811_7f89_4341_bf13_65381da1e379.slice/crio-44fb390604eb7444743d5e5a3b3363d7000a5a652a087d0e58942896f9049706 WatchSource:0}: Error finding container 44fb390604eb7444743d5e5a3b3363d7000a5a652a087d0e58942896f9049706: Status 404 returned error can't find the container with id 44fb390604eb7444743d5e5a3b3363d7000a5a652a087d0e58942896f9049706
Oct 30 20:08:29.767234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.767207    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh"
Oct 30 20:08:29.805391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.805354    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-dcfxs" event={"ID":"7bfb1811-7f89-4341-bf13-65381da1e379","Type":"ContainerStarted","Data":"caea4da5676d041556c7cd8ceebb80bdc125a152f4355e000a27282eb50d2b2b"}
Oct 30 20:08:29.805551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.805401    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-dcfxs" event={"ID":"7bfb1811-7f89-4341-bf13-65381da1e379","Type":"ContainerStarted","Data":"44fb390604eb7444743d5e5a3b3363d7000a5a652a087d0e58942896f9049706"}
Oct 30 20:08:29.827710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:29.827666    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-dcfxs" podStartSLOduration=1.8276532570000001 podStartE2EDuration="1.827653257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:29.827205789 +0000 UTC m=+230.026778720" watchObservedRunningTime="2026-10-30 20:08:29.827653257 +0000 UTC m=+230.027226182"
Oct 30 20:08:32.856685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:32.856653    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:35.122955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:35.122922    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:35.224463 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:35.224432    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:42.173678 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:42.173645    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:42.281676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:42.281644    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:44.018009 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.017979    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:44.532570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.532540    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"]
Oct 30 20:08:44.538092 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.538071    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.550172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.550153    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-5rwfr\""
Oct 30 20:08:44.553166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.553113    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"]
Oct 30 20:08:44.555194 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.555175    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 20:08:44.555318 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.555301    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 20:08:44.638345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.638318    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb67r\" (UniqueName: \"kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.638481 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.638361    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.638481 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.638456    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.739657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.739628    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.739762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.739672    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb67r\" (UniqueName: \"kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.739762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.739702    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.739993 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.739976    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.740035 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.740004    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.753313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.753287    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb67r\" (UniqueName: \"kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:44.851362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:44.851298    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:45.122917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:45.122845    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"]
Oct 30 20:08:45.125644 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:45.125612    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1320b1e_10d4_47f1_9457_6747fa61d739.slice/crio-0c0dd8cef1f1cd5e5d275d1be76755bdc17760e723fab63144513a7ccc33f977 WatchSource:0}: Error finding container 0c0dd8cef1f1cd5e5d275d1be76755bdc17760e723fab63144513a7ccc33f977: Status 404 returned error can't find the container with id 0c0dd8cef1f1cd5e5d275d1be76755bdc17760e723fab63144513a7ccc33f977
Oct 30 20:08:45.870998 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:45.870959    6389 generic.go:356] "Generic (PLEG): container finished" podID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerID="ee428c089e73459ccda504bab682e6fbdcfd11041d45b5fd8578d87b52adcfb9" exitCode=0
Oct 30 20:08:45.871197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:45.871041    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95" event={"ID":"b1320b1e-10d4-47f1-9457-6747fa61d739","Type":"ContainerDied","Data":"ee428c089e73459ccda504bab682e6fbdcfd11041d45b5fd8578d87b52adcfb9"}
Oct 30 20:08:45.871197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:45.871097    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95" event={"ID":"b1320b1e-10d4-47f1-9457-6747fa61d739","Type":"ContainerStarted","Data":"0c0dd8cef1f1cd5e5d275d1be76755bdc17760e723fab63144513a7ccc33f977"}
Oct 30 20:08:46.876446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:46.876414    6389 generic.go:356] "Generic (PLEG): container finished" podID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerID="6fcd090a2e2f09dc6c8cd0aaf3ea47ebc89a83d432c3b4c526392cf87a0410d7" exitCode=0
Oct 30 20:08:46.876835 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:46.876507    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95" event={"ID":"b1320b1e-10d4-47f1-9457-6747fa61d739","Type":"ContainerDied","Data":"6fcd090a2e2f09dc6c8cd0aaf3ea47ebc89a83d432c3b4c526392cf87a0410d7"}
Oct 30 20:08:47.882678 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:47.882642    6389 generic.go:356] "Generic (PLEG): container finished" podID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerID="c1562c5866c3e449296cdbdcb1d5be1dbf753e5c1c2bb7874ee35030fa5352fa" exitCode=0
Oct 30 20:08:47.883068 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:47.882687    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95" event={"ID":"b1320b1e-10d4-47f1-9457-6747fa61d739","Type":"ContainerDied","Data":"c1562c5866c3e449296cdbdcb1d5be1dbf753e5c1c2bb7874ee35030fa5352fa"}
Oct 30 20:08:49.002339 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.002315    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:49.074476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.074443    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r\" (UniqueName: \"kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r\") pod \"b1320b1e-10d4-47f1-9457-6747fa61d739\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") "
Oct 30 20:08:49.074634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.074549    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util\") pod \"b1320b1e-10d4-47f1-9457-6747fa61d739\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") "
Oct 30 20:08:49.074634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.074582    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle\") pod \"b1320b1e-10d4-47f1-9457-6747fa61d739\" (UID: \"b1320b1e-10d4-47f1-9457-6747fa61d739\") "
Oct 30 20:08:49.075159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.075133    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle" pod "b1320b1e-10d4-47f1-9457-6747fa61d739" (UID: "b1320b1e-10d4-47f1-9457-6747fa61d739"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:49.076484 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.076464    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r" pod "b1320b1e-10d4-47f1-9457-6747fa61d739" (UID: "b1320b1e-10d4-47f1-9457-6747fa61d739"). InnerVolumeSpecName "kube-api-access-lb67r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:49.079898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.079876    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util" pod "b1320b1e-10d4-47f1-9457-6747fa61d739" (UID: "b1320b1e-10d4-47f1-9457-6747fa61d739"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:49.175448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.175351    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:49.175448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.175393    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1320b1e-10d4-47f1-9457-6747fa61d739-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:49.175448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.175411    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb67r\" (UniqueName: \"kubernetes.io/projected/b1320b1e-10d4-47f1-9457-6747fa61d739-kube-api-access-lb67r\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:49.892514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.892475    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95" event={"ID":"b1320b1e-10d4-47f1-9457-6747fa61d739","Type":"ContainerDied","Data":"0c0dd8cef1f1cd5e5d275d1be76755bdc17760e723fab63144513a7ccc33f977"}
Oct 30 20:08:49.892723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.892523    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71gjq95"
Oct 30 20:08:49.892723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:49.892527    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c0dd8cef1f1cd5e5d275d1be76755bdc17760e723fab63144513a7ccc33f977"
Oct 30 20:08:54.789855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:54.789813    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:55.474829 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.474796    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"]
Oct 30 20:08:55.475088 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475077    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="util"
Oct 30 20:08:55.475169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475090    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="util"
Oct 30 20:08:55.475169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475097    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="extract"
Oct 30 20:08:55.475169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475102    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="extract"
Oct 30 20:08:55.475169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475127    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="pull"
Oct 30 20:08:55.475169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475133    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="pull"
Oct 30 20:08:55.475315 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.475182    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1320b1e-10d4-47f1-9457-6747fa61d739" containerName="extract"
Oct 30 20:08:55.477067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.477051    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.488782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.488688    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 20:08:55.488782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.488704    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 20:08:55.488782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.488755    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d9p54\""
Oct 30 20:08:55.488993 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.488790    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 20:08:55.508141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.508101    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"]
Oct 30 20:08:55.524398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.524376    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x52ht\" (UniqueName: \"kubernetes.io/projected/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-kube-api-access-x52ht\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.524519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.524409    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-tmp\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.524519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.524437    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.524588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.524555    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-metrics-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.524626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.524584    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-manifests\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.524626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.524605    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.568263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.568238    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 20:08:55.568480 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.568462    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 20:08:55.625613 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.625583    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x52ht\" (UniqueName: \"kubernetes.io/projected/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-kube-api-access-x52ht\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.625764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.625620    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-tmp\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.625764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.625663    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.625764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.625700    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-metrics-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.625764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.625740    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-manifests\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.625764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.625762    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.626140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.626096    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-manifests\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.628192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.628165    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-tmp\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.628286 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.628269    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.628345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.628268    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-metrics-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.628397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.628376    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.638868 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.638840    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x52ht\" (UniqueName: \"kubernetes.io/projected/9f23eedf-3687-4fdf-a3e1-27d2455e0dbc-kube-api-access-x52ht\") pod \"opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5\" (UID: \"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.789568 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:55.789486    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:08:55.789715 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:08:55.789575    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:08:56.067237 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:56.067153    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:56.088314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:56.088249    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"]
Oct 30 20:08:56.090277 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:08:56.090255    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f23eedf_3687_4fdf_a3e1_27d2455e0dbc.slice/crio-1814945445a3c463c766288001dd7f0cb455bc168fff868a9d5b933c24028df5 WatchSource:0}: Error finding container 1814945445a3c463c766288001dd7f0cb455bc168fff868a9d5b933c24028df5: Status 404 returned error can't find the container with id 1814945445a3c463c766288001dd7f0cb455bc168fff868a9d5b933c24028df5
Oct 30 20:08:56.228700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:56.228663    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:08:56.920187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:56.920141    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5" event={"ID":"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc","Type":"ContainerStarted","Data":"1814945445a3c463c766288001dd7f0cb455bc168fff868a9d5b933c24028df5"}
Oct 30 20:08:59.934414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:59.934333    6389 generic.go:356] "Generic (PLEG): container finished" podID="9f23eedf-3687-4fdf-a3e1-27d2455e0dbc" containerID="2709c9f14567e6816e19961c0f9c605a86e1e39759d09fff1a02d588a3d5e526" exitCode=0
Oct 30 20:08:59.934786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:08:59.934418    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5" event={"ID":"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc","Type":"ContainerDied","Data":"2709c9f14567e6816e19961c0f9c605a86e1e39759d09fff1a02d588a3d5e526"}
Oct 30 20:09:00.202003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:00.201920    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:00.340030 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:00.340006    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:00.946850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:00.946806    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5" event={"ID":"9f23eedf-3687-4fdf-a3e1-27d2455e0dbc","Type":"ContainerStarted","Data":"824749f426e9413ec62361c33d2a0c403be3b09341380e7608ca34fbfa46dd3a"}
Oct 30 20:09:00.947368 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:00.947162    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:09:00.993100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:00.993050    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5" podStartSLOduration=2.208158229 podStartE2EDuration="5.993036394s" totalImagesPullingTime="3.293527553s" totalInitContainerRuntime="491.350612ms" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:00.992632257 +0000 UTC m=+261.192205187" watchObservedRunningTime="2026-10-30 20:09:00.993036394 +0000 UTC m=+261.192609319"
Oct 30 20:09:11.952897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:11.952869    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5"
Oct 30 20:09:14.425027 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:14.424997    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:14.579960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:14.579927    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:15.850552 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:15.850518    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:16.051639 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.051607    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-observability-584f99c788-p6lsk"]
Oct 30 20:09:16.058956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.058930    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.067656 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.067630    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-webhook-cert\""
Oct 30 20:09:16.067795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.067657    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-dockercfg-z7g2l\""
Oct 30 20:09:16.085610 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.085586    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-p6lsk"]
Oct 30 20:09:16.143933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.143870    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:16.198665 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.198634    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/35f63dc5-4733-491e-878a-2b30584bcd2a-webhook-certs\") pod \"odh-observability-584f99c788-p6lsk\" (UID: \"35f63dc5-4733-491e-878a-2b30584bcd2a\") " pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.198816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.198692    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf8xg\" (UniqueName: \"kubernetes.io/projected/35f63dc5-4733-491e-878a-2b30584bcd2a-kube-api-access-bf8xg\") pod \"odh-observability-584f99c788-p6lsk\" (UID: \"35f63dc5-4733-491e-878a-2b30584bcd2a\") " pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.299163 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.299132    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf8xg\" (UniqueName: \"kubernetes.io/projected/35f63dc5-4733-491e-878a-2b30584bcd2a-kube-api-access-bf8xg\") pod \"odh-observability-584f99c788-p6lsk\" (UID: \"35f63dc5-4733-491e-878a-2b30584bcd2a\") " pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.299323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.299198    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/35f63dc5-4733-491e-878a-2b30584bcd2a-webhook-certs\") pod \"odh-observability-584f99c788-p6lsk\" (UID: \"35f63dc5-4733-491e-878a-2b30584bcd2a\") " pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.301659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.301629    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/35f63dc5-4733-491e-878a-2b30584bcd2a-webhook-certs\") pod \"odh-observability-584f99c788-p6lsk\" (UID: \"35f63dc5-4733-491e-878a-2b30584bcd2a\") " pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.323798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.323770    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf8xg\" (UniqueName: \"kubernetes.io/projected/35f63dc5-4733-491e-878a-2b30584bcd2a-kube-api-access-bf8xg\") pod \"odh-observability-584f99c788-p6lsk\" (UID: \"35f63dc5-4733-491e-878a-2b30584bcd2a\") " pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.329781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.329755    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:16.371957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.371933    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:16.654772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.654679    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-p6lsk"]
Oct 30 20:09:16.657746 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:09:16.657718    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f63dc5_4733_491e_878a_2b30584bcd2a.slice/crio-253d20cccff72a7a3c0c64bce06dca45814484462b870545b7f67cc427e4a6c5 WatchSource:0}: Error finding container 253d20cccff72a7a3c0c64bce06dca45814484462b870545b7f67cc427e4a6c5: Status 404 returned error can't find the container with id 253d20cccff72a7a3c0c64bce06dca45814484462b870545b7f67cc427e4a6c5
Oct 30 20:09:16.669051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.668996    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"]
Oct 30 20:09:16.673477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.673458    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.682523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.682494    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"]
Oct 30 20:09:16.772568 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.772543    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:09:16.772719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.772578    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 20:09:16.772719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.772642    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 20:09:16.772719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.772655    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 20:09:16.772719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.772719    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 20:09:16.772857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.772729    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h9wrx\""
Oct 30 20:09:16.802764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.802732    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/659a2fc2-a8d4-4712-85cb-68133288c079-metrics-cert\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.802764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.802762    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/659a2fc2-a8d4-4712-85cb-68133288c079-cert\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.802970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.802787    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/659a2fc2-a8d4-4712-85cb-68133288c079-manager-config\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.802970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.802803    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwnkm\" (UniqueName: \"kubernetes.io/projected/659a2fc2-a8d4-4712-85cb-68133288c079-kube-api-access-bwnkm\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.904328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.904297    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/659a2fc2-a8d4-4712-85cb-68133288c079-manager-config\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.904328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.904330    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwnkm\" (UniqueName: \"kubernetes.io/projected/659a2fc2-a8d4-4712-85cb-68133288c079-kube-api-access-bwnkm\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.904845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.904484    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/659a2fc2-a8d4-4712-85cb-68133288c079-metrics-cert\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.904845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.904553    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/659a2fc2-a8d4-4712-85cb-68133288c079-cert\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.905061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.905039    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/659a2fc2-a8d4-4712-85cb-68133288c079-manager-config\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.906902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.906882    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/659a2fc2-a8d4-4712-85cb-68133288c079-cert\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.907037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.906944    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/659a2fc2-a8d4-4712-85cb-68133288c079-metrics-cert\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.917756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.917733    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwnkm\" (UniqueName: \"kubernetes.io/projected/659a2fc2-a8d4-4712-85cb-68133288c079-kube-api-access-bwnkm\") pod \"lws-controller-manager-8656d6df85-2xztw\" (UID: \"659a2fc2-a8d4-4712-85cb-68133288c079\") " pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:16.986434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:16.986410    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:17.015951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:17.015916    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-p6lsk" event={"ID":"35f63dc5-4733-491e-878a-2b30584bcd2a","Type":"ContainerStarted","Data":"253d20cccff72a7a3c0c64bce06dca45814484462b870545b7f67cc427e4a6c5"}
Oct 30 20:09:17.264437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:17.263990    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"]
Oct 30 20:09:18.021528 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:18.021488    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw" event={"ID":"659a2fc2-a8d4-4712-85cb-68133288c079","Type":"ContainerStarted","Data":"35f1973ecf066348f9e04731db145adf107ee64592d959442e7a530bd868b131"}
Oct 30 20:09:19.028637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:19.028602    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-p6lsk" event={"ID":"35f63dc5-4733-491e-878a-2b30584bcd2a","Type":"ContainerStarted","Data":"dfd9a6bd4a4682dfda6901bd93860a56c65cfc98439bc18d3043827e2e2402ce"}
Oct 30 20:09:19.029103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:19.028797    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:19.050774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:19.050711    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-observability-584f99c788-p6lsk" podStartSLOduration=0.80488966 podStartE2EDuration="3.050693107s" totalImagesPullingTime="2.245803447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:19.050028203 +0000 UTC m=+279.249601133" watchObservedRunningTime="2026-10-30 20:09:19.050693107 +0000 UTC m=+279.250266036"
Oct 30 20:09:20.035014 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:20.034963    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw" event={"ID":"659a2fc2-a8d4-4712-85cb-68133288c079","Type":"ContainerStarted","Data":"3db4f1cc1a7177b68836191c113d84ab8e309e0c6b4bd716fb8d1495498ab9e7"}
Oct 30 20:09:20.035523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:20.035050    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:20.054959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:20.054910    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw" podStartSLOduration=1.943536997 podStartE2EDuration="4.054895219s" totalImagesPullingTime="2.111358222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:20.0539076 +0000 UTC m=+280.253480534" watchObservedRunningTime="2026-10-30 20:09:20.054895219 +0000 UTC m=+280.254468144"
Oct 30 20:09:25.204701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:25.204659    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:25.323275 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:25.323241    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:25.792217 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:09:25.792179    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:09:26.861635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:26.861593    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:26.977058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:26.977023    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:27.139723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.139640    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:27.157234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.157205    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"]
Oct 30 20:09:27.160626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.160596    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.174383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.174362    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"]
Oct 30 20:09:27.179641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.179618    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 20:09:27.179884 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.179868    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-gdcsw\""
Oct 30 20:09:27.203391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.203364    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0b46f6b-b636-48f0-8bb6-822eee052780-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.203535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.203423    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e0b46f6b-b636-48f0-8bb6-822eee052780-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.203535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.203442    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0b46f6b-b636-48f0-8bb6-822eee052780-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.203535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.203462    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72q9x\" (UniqueName: \"kubernetes.io/projected/e0b46f6b-b636-48f0-8bb6-822eee052780-kube-api-access-72q9x\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.244831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.244802    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:27.304904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.304872    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e0b46f6b-b636-48f0-8bb6-822eee052780-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.304904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.304904    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0b46f6b-b636-48f0-8bb6-822eee052780-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.305148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.304926    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72q9x\" (UniqueName: \"kubernetes.io/projected/e0b46f6b-b636-48f0-8bb6-822eee052780-kube-api-access-72q9x\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.305148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.304976    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0b46f6b-b636-48f0-8bb6-822eee052780-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.305422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.305399    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0b46f6b-b636-48f0-8bb6-822eee052780-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.305495 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.305466    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0b46f6b-b636-48f0-8bb6-822eee052780-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.306175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.306156    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e0b46f6b-b636-48f0-8bb6-822eee052780-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.318351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.318329    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72q9x\" (UniqueName: \"kubernetes.io/projected/e0b46f6b-b636-48f0-8bb6-822eee052780-kube-api-access-72q9x\") pod \"kserve-module-controller-manager-5cfb77bf8c-xv76h\" (UID: \"e0b46f6b-b636-48f0-8bb6-822eee052780\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.473088 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.473059    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:27.753758 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:27.753681    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"]
Oct 30 20:09:27.756631 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:09:27.756602    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b46f6b_b636_48f0_8bb6_822eee052780.slice/crio-5b21d33f82bb6d1dee46645ddda3baefa5783f7936db3989a544be4d0fda6dcd WatchSource:0}: Error finding container 5b21d33f82bb6d1dee46645ddda3baefa5783f7936db3989a544be4d0fda6dcd: Status 404 returned error can't find the container with id 5b21d33f82bb6d1dee46645ddda3baefa5783f7936db3989a544be4d0fda6dcd
Oct 30 20:09:28.069901 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:28.069817    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h" event={"ID":"e0b46f6b-b636-48f0-8bb6-822eee052780","Type":"ContainerStarted","Data":"5b21d33f82bb6d1dee46645ddda3baefa5783f7936db3989a544be4d0fda6dcd"}
Oct 30 20:09:28.226025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:28.225987    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:30.037796 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:30.037765    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-observability-584f99c788-p6lsk"
Oct 30 20:09:30.659689 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:30.659656    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:30.762768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:30.762740    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:31.042081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:31.042048    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8656d6df85-2xztw"
Oct 30 20:09:31.085211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:31.085178    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h" event={"ID":"e0b46f6b-b636-48f0-8bb6-822eee052780","Type":"ContainerStarted","Data":"852f6db1d082e1f420f2b5efdeadef646743aed0e3d5b95f235a20b12001a55d"}
Oct 30 20:09:31.085401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:31.085287    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:31.873366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:31.873329    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:32.072882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:32.072845    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:32.172628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:32.172520    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:32.911786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:32.911740    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:09:39.947029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:39.946996    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:09:39.947560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:39.947541    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:09:52.121585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:52.121547    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h"
Oct 30 20:09:52.150545 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:09:52.150480    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-xv76h" podStartSLOduration=22.776439897 podStartE2EDuration="25.150461939s" totalImagesPullingTime="2.374022042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:31.11200429 +0000 UTC m=+291.311577217" watchObservedRunningTime="2026-10-30 20:09:52.150461939 +0000 UTC m=+312.350034869"
Oct 30 20:09:55.790973 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:09:55.790938    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:10:13.143434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.143388    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:13.255363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.255329    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"]
Oct 30 20:10:13.268167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.268141    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.273447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.273417    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"]
Oct 30 20:10:13.276930 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.276898    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:13.278619 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.278595    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 30 20:10:13.278742 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.278666    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xckks\""
Oct 30 20:10:13.311592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.311561    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-2wfgc\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.311592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.311594    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lzr\" (UniqueName: \"kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr\") pod \"llmisvc-controller-manager-869d5dcc5d-2wfgc\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.412335 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.412260    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-2wfgc\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.412335 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.412291    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lzr\" (UniqueName: \"kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr\") pod \"llmisvc-controller-manager-869d5dcc5d-2wfgc\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.415039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.415016    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-2wfgc\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.424438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.424413    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:13.425375 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.425359    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lzr\" (UniqueName: \"kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr\") pod \"llmisvc-controller-manager-869d5dcc5d-2wfgc\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.530345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.530317    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:13.582325 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.582298    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:13.855670 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:13.855538    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"]
Oct 30 20:10:13.858173 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:10:13.858138    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6e852c03_a5bd_44b5_8792_ef2096c89a3f.slice/crio-24d8a9e4de2e01fa09993568356be115e9bc081bb2237d679b6144240351cc35 WatchSource:0}: Error finding container 24d8a9e4de2e01fa09993568356be115e9bc081bb2237d679b6144240351cc35: Status 404 returned error can't find the container with id 24d8a9e4de2e01fa09993568356be115e9bc081bb2237d679b6144240351cc35
Oct 30 20:10:14.268754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:14.268714    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc" event={"ID":"6e852c03-a5bd-44b5-8792-ef2096c89a3f","Type":"ContainerStarted","Data":"24d8a9e4de2e01fa09993568356be115e9bc081bb2237d679b6144240351cc35"}
Oct 30 20:10:14.644655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:14.644568    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:14.828593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:14.828559    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:14.975367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:14.975335    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:16.278540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:16.278503    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc" event={"ID":"6e852c03-a5bd-44b5-8792-ef2096c89a3f","Type":"ContainerStarted","Data":"df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405"}
Oct 30 20:10:16.278940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:16.278602    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:16.298922 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:16.298880    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc" podStartSLOduration=1.486250949 podStartE2EDuration="3.298868236s" totalImagesPullingTime="1.812617287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:10:16.297430934 +0000 UTC m=+336.497003862" watchObservedRunningTime="2026-10-30 20:10:16.298868236 +0000 UTC m=+336.498441164"
Oct 30 20:10:25.790434 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:10:25.790395    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:10:32.975449 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:32.975411    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:10:47.285719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:10:47.285681    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:10:55.790674 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:10:55.790594    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:11:08.067762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:08.067726    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:08.169943 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:08.169893    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:10.835345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.835308    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"]
Oct 30 20:11:10.838880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.838859    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.848948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.848923    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 20:11:10.849084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.848924    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 20:11:10.849084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.849078    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-jzprn\""
Oct 30 20:11:10.854215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.854189    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"]
Oct 30 20:11:10.889022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.888990    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.889219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.889028    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.889219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.889194    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmhh2\" (UniqueName: \"kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.932870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.932834    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"]
Oct 30 20:11:10.936924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.936902    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:10.941236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.941204    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-gmxmj\""
Oct 30 20:11:10.947893 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.947874    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"]
Oct 30 20:11:10.990489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990458    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:10.990654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990505    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhh2\" (UniqueName: \"kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.990654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990579    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq9tf\" (UniqueName: \"kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:10.990654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990615    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.990654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990644    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.990904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990666    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:10.991016 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.990998    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:10.991100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:10.991081    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:11.004130 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.004087    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmhh2\" (UniqueName: \"kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:11.031155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.031102    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"]
Oct 30 20:11:11.035299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.035280    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.040300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.040280    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-wmdvk\""
Oct 30 20:11:11.049994 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.049499    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"]
Oct 30 20:11:11.092306 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092224    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.092306 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092267    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.092477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092346    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5v2j\" (UniqueName: \"kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.092477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092411    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.092477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092442    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.092590 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092554    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq9tf\" (UniqueName: \"kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.092779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092757    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.092779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.092772    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.105583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.105552    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq9tf\" (UniqueName: \"kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.138546 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.138517    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"]
Oct 30 20:11:11.142388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.142369    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.147279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.147259    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-44w8j\""
Oct 30 20:11:11.157282 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.157260    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:11.160026 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.159998    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"]
Oct 30 20:11:11.193784 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.193745    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5v2j\" (UniqueName: \"kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.193955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.193798    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.193955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.193850    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.194065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.193961    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.194065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.194010    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x7xd\" (UniqueName: \"kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.194201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.194079    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.194334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.194312    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.194392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.194376    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.207401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.207367    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5v2j\" (UniqueName: \"kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.256618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.256588    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:11.295477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.295444    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.295651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.295548    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x7xd\" (UniqueName: \"kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.295651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.295576    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.295936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.295915    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.296008 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.295915    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.314534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.314506    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x7xd\" (UniqueName: \"kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.359426 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.359355    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:11.439213 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.439183    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"]
Oct 30 20:11:11.441582 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:11:11.441556    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94c7047_1900_4bac_89fe_2f7431435587.slice/crio-f5c92518bfda10ddae6cbc5081ab2bde7932ef2b10f5723206c8947b62c86649 WatchSource:0}: Error finding container f5c92518bfda10ddae6cbc5081ab2bde7932ef2b10f5723206c8947b62c86649: Status 404 returned error can't find the container with id f5c92518bfda10ddae6cbc5081ab2bde7932ef2b10f5723206c8947b62c86649
Oct 30 20:11:11.461167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.461146    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:11.511010 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.510972    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k" event={"ID":"b94c7047-1900-4bac-89fe-2f7431435587","Type":"ContainerStarted","Data":"f5c92518bfda10ddae6cbc5081ab2bde7932ef2b10f5723206c8947b62c86649"}
Oct 30 20:11:11.533679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.533642    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"]
Oct 30 20:11:11.538834 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:11:11.538797    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ff46bd_4e37_4e9a_9258_7c4e3e91399a.slice/crio-26c38e2307958135331aadd326e5213814625888ff9dbc17462b8a242e871427 WatchSource:0}: Error finding container 26c38e2307958135331aadd326e5213814625888ff9dbc17462b8a242e871427: Status 404 returned error can't find the container with id 26c38e2307958135331aadd326e5213814625888ff9dbc17462b8a242e871427
Oct 30 20:11:11.633085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.632963    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"]
Oct 30 20:11:11.635650 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:11:11.635623    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462e0d54_1bec_4e67_a243_72d25ffe8b1d.slice/crio-d16aec85e309061ecaea10b9cc62ad370f39a60000c3efce1a32d13692c9e786 WatchSource:0}: Error finding container d16aec85e309061ecaea10b9cc62ad370f39a60000c3efce1a32d13692c9e786: Status 404 returned error can't find the container with id d16aec85e309061ecaea10b9cc62ad370f39a60000c3efce1a32d13692c9e786
Oct 30 20:11:11.740982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:11.740953    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"]
Oct 30 20:11:11.799879 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:11:11.799843    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a20e2b_4dc3_4a1a_9d65_6bd54f22e024.slice/crio-a4bed79a53db1d534875f3983499cd1092616a704c63a4233ffde9e329d5a853 WatchSource:0}: Error finding container a4bed79a53db1d534875f3983499cd1092616a704c63a4233ffde9e329d5a853: Status 404 returned error can't find the container with id a4bed79a53db1d534875f3983499cd1092616a704c63a4233ffde9e329d5a853
Oct 30 20:11:12.516329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.516297    6389 generic.go:356] "Generic (PLEG): container finished" podID="b94c7047-1900-4bac-89fe-2f7431435587" containerID="8bf77fec670b0bebf8289dc257793cb2355163724cf9037d762b14a7116efa72" exitCode=0
Oct 30 20:11:12.516708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.516384    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k" event={"ID":"b94c7047-1900-4bac-89fe-2f7431435587","Type":"ContainerDied","Data":"8bf77fec670b0bebf8289dc257793cb2355163724cf9037d762b14a7116efa72"}
Oct 30 20:11:12.518023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.517997    6389 generic.go:356] "Generic (PLEG): container finished" podID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerID="162818542b045c252ff956cfa369acbc29751e638b7cb473c07595a587399ef0" exitCode=0
Oct 30 20:11:12.518166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.518033    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j" event={"ID":"462e0d54-1bec-4e67-a243-72d25ffe8b1d","Type":"ContainerDied","Data":"162818542b045c252ff956cfa369acbc29751e638b7cb473c07595a587399ef0"}
Oct 30 20:11:12.518166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.518098    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j" event={"ID":"462e0d54-1bec-4e67-a243-72d25ffe8b1d","Type":"ContainerStarted","Data":"d16aec85e309061ecaea10b9cc62ad370f39a60000c3efce1a32d13692c9e786"}
Oct 30 20:11:12.519571 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.519553    6389 generic.go:356] "Generic (PLEG): container finished" podID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerID="cfbf8a9d070b79f933d028d676196ac33cd354d1d73a30d70bf96f739ba3c37c" exitCode=0
Oct 30 20:11:12.519694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.519668    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r" event={"ID":"82ff46bd-4e37-4e9a-9258-7c4e3e91399a","Type":"ContainerDied","Data":"cfbf8a9d070b79f933d028d676196ac33cd354d1d73a30d70bf96f739ba3c37c"}
Oct 30 20:11:12.519947 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.519713    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r" event={"ID":"82ff46bd-4e37-4e9a-9258-7c4e3e91399a","Type":"ContainerStarted","Data":"26c38e2307958135331aadd326e5213814625888ff9dbc17462b8a242e871427"}
Oct 30 20:11:12.521355 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.521294    6389 generic.go:356] "Generic (PLEG): container finished" podID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerID="79af45b92d2000ded236582c56d361052692e5706669bb6e1201036400fde4d9" exitCode=0
Oct 30 20:11:12.521355 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.521318    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf" event={"ID":"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024","Type":"ContainerDied","Data":"79af45b92d2000ded236582c56d361052692e5706669bb6e1201036400fde4d9"}
Oct 30 20:11:12.521355 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:12.521347    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf" event={"ID":"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024","Type":"ContainerStarted","Data":"a4bed79a53db1d534875f3983499cd1092616a704c63a4233ffde9e329d5a853"}
Oct 30 20:11:14.532781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:14.532748    6389 generic.go:356] "Generic (PLEG): container finished" podID="b94c7047-1900-4bac-89fe-2f7431435587" containerID="84bfb29a2950ae4bdb2045f8c655bc4e9682093d328b38a6847a77526a8af0d3" exitCode=0
Oct 30 20:11:14.533231 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:14.532811    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k" event={"ID":"b94c7047-1900-4bac-89fe-2f7431435587","Type":"ContainerDied","Data":"84bfb29a2950ae4bdb2045f8c655bc4e9682093d328b38a6847a77526a8af0d3"}
Oct 30 20:11:14.534840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:14.534813    6389 generic.go:356] "Generic (PLEG): container finished" podID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerID="0ece86ce10f6ea73ee6500f8eb196c58d397d8425f44e8c71883bb86e98b7ef4" exitCode=0
Oct 30 20:11:14.534940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:14.534886    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j" event={"ID":"462e0d54-1bec-4e67-a243-72d25ffe8b1d","Type":"ContainerDied","Data":"0ece86ce10f6ea73ee6500f8eb196c58d397d8425f44e8c71883bb86e98b7ef4"}
Oct 30 20:11:14.537102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:14.537080    6389 generic.go:356] "Generic (PLEG): container finished" podID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerID="92c703902a05a2004ee14fa1c73ba9b35b784389e18456631552b726d4577587" exitCode=0
Oct 30 20:11:14.537213 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:14.537136    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r" event={"ID":"82ff46bd-4e37-4e9a-9258-7c4e3e91399a","Type":"ContainerDied","Data":"92c703902a05a2004ee14fa1c73ba9b35b784389e18456631552b726d4577587"}
Oct 30 20:11:15.543632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.543598    6389 generic.go:356] "Generic (PLEG): container finished" podID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerID="2578df940b771d881e4db691b77eb36b4738472fedae491d391fcb4646cc5a81" exitCode=0
Oct 30 20:11:15.544083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.543683    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf" event={"ID":"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024","Type":"ContainerDied","Data":"2578df940b771d881e4db691b77eb36b4738472fedae491d391fcb4646cc5a81"}
Oct 30 20:11:15.546160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.546137    6389 generic.go:356] "Generic (PLEG): container finished" podID="b94c7047-1900-4bac-89fe-2f7431435587" containerID="3142e51e8f5db9080f361312849055d1e8b1cad6b47e5065fb7de7e049841f5a" exitCode=0
Oct 30 20:11:15.546262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.546215    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k" event={"ID":"b94c7047-1900-4bac-89fe-2f7431435587","Type":"ContainerDied","Data":"3142e51e8f5db9080f361312849055d1e8b1cad6b47e5065fb7de7e049841f5a"}
Oct 30 20:11:15.548485 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.548462    6389 generic.go:356] "Generic (PLEG): container finished" podID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerID="7ea64463283bcdb9a7eac64d75a44049f931e82e9c233dbe53b9100b6fdb07c1" exitCode=0
Oct 30 20:11:15.548585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.548548    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j" event={"ID":"462e0d54-1bec-4e67-a243-72d25ffe8b1d","Type":"ContainerDied","Data":"7ea64463283bcdb9a7eac64d75a44049f931e82e9c233dbe53b9100b6fdb07c1"}
Oct 30 20:11:15.550636 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.550615    6389 generic.go:356] "Generic (PLEG): container finished" podID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerID="abac8b8bb810dbb40ffff49a2cf8689f52eb5dd2e7bc8d4b2cd6df008257abb0" exitCode=0
Oct 30 20:11:15.550740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:15.550679    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r" event={"ID":"82ff46bd-4e37-4e9a-9258-7c4e3e91399a","Type":"ContainerDied","Data":"abac8b8bb810dbb40ffff49a2cf8689f52eb5dd2e7bc8d4b2cd6df008257abb0"}
Oct 30 20:11:16.557730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.557695    6389 generic.go:356] "Generic (PLEG): container finished" podID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerID="efd017cab83a2e6de6b25bb78642b913dc5d3e0446e0c4c2165cc09565b3c59b" exitCode=0
Oct 30 20:11:16.558249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.557754    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf" event={"ID":"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024","Type":"ContainerDied","Data":"efd017cab83a2e6de6b25bb78642b913dc5d3e0446e0c4c2165cc09565b3c59b"}
Oct 30 20:11:16.695505 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.695479    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:16.744628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.744606    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:16.746699 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.746680    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle\") pod \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") "
Oct 30 20:11:16.746815 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.746730    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf\" (UniqueName: \"kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf\") pod \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") "
Oct 30 20:11:16.746815 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.746782    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util\") pod \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\" (UID: \"82ff46bd-4e37-4e9a-9258-7c4e3e91399a\") "
Oct 30 20:11:16.747230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.747208    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle" pod "82ff46bd-4e37-4e9a-9258-7c4e3e91399a" (UID: "82ff46bd-4e37-4e9a-9258-7c4e3e91399a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:16.747406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.747385    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:16.748843 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.748820    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf" pod "82ff46bd-4e37-4e9a-9258-7c4e3e91399a" (UID: "82ff46bd-4e37-4e9a-9258-7c4e3e91399a"). InnerVolumeSpecName "kube-api-access-vq9tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:11:16.752164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.752140    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util" pod "82ff46bd-4e37-4e9a-9258-7c4e3e91399a" (UID: "82ff46bd-4e37-4e9a-9258-7c4e3e91399a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:16.847284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847213    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle\") pod \"b94c7047-1900-4bac-89fe-2f7431435587\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") "
Oct 30 20:11:16.847284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847248    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util\") pod \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") "
Oct 30 20:11:16.847284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847273    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle\") pod \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") "
Oct 30 20:11:16.847531 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847316    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util\") pod \"b94c7047-1900-4bac-89fe-2f7431435587\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") "
Oct 30 20:11:16.847531 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847344    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j\" (UniqueName: \"kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j\") pod \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\" (UID: \"462e0d54-1bec-4e67-a243-72d25ffe8b1d\") "
Oct 30 20:11:16.847531 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847368    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2\" (UniqueName: \"kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2\") pod \"b94c7047-1900-4bac-89fe-2f7431435587\" (UID: \"b94c7047-1900-4bac-89fe-2f7431435587\") "
Oct 30 20:11:16.847686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847597    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.847686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847617    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.847686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847631    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq9tf\" (UniqueName: \"kubernetes.io/projected/82ff46bd-4e37-4e9a-9258-7c4e3e91399a-kube-api-access-vq9tf\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.847844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847777    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle" pod "b94c7047-1900-4bac-89fe-2f7431435587" (UID: "b94c7047-1900-4bac-89fe-2f7431435587"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:16.847844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.847781    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle" pod "462e0d54-1bec-4e67-a243-72d25ffe8b1d" (UID: "462e0d54-1bec-4e67-a243-72d25ffe8b1d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:16.849455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.849427    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j" pod "462e0d54-1bec-4e67-a243-72d25ffe8b1d" (UID: "462e0d54-1bec-4e67-a243-72d25ffe8b1d"). InnerVolumeSpecName "kube-api-access-q5v2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:11:16.849574 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.849550    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2" pod "b94c7047-1900-4bac-89fe-2f7431435587" (UID: "b94c7047-1900-4bac-89fe-2f7431435587"). InnerVolumeSpecName "kube-api-access-hmhh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:11:16.852866 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.852832    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util" pod "462e0d54-1bec-4e67-a243-72d25ffe8b1d" (UID: "462e0d54-1bec-4e67-a243-72d25ffe8b1d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:16.853172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.853154    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util" pod "b94c7047-1900-4bac-89fe-2f7431435587" (UID: "b94c7047-1900-4bac-89fe-2f7431435587"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:16.948215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.948178    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.948215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.948210    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.948215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.948221    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462e0d54-1bec-4e67-a243-72d25ffe8b1d-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.948215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.948230    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94c7047-1900-4bac-89fe-2f7431435587-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.948438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.948241    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5v2j\" (UniqueName: \"kubernetes.io/projected/462e0d54-1bec-4e67-a243-72d25ffe8b1d-kube-api-access-q5v2j\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:16.948438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:16.948251    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmhh2\" (UniqueName: \"kubernetes.io/projected/b94c7047-1900-4bac-89fe-2f7431435587-kube-api-access-hmhh2\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:17.564269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.564229    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k" event={"ID":"b94c7047-1900-4bac-89fe-2f7431435587","Type":"ContainerDied","Data":"f5c92518bfda10ddae6cbc5081ab2bde7932ef2b10f5723206c8947b62c86649"}
Oct 30 20:11:17.564755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.564289    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72zb6k"
Oct 30 20:11:17.564755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.564314    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5c92518bfda10ddae6cbc5081ab2bde7932ef2b10f5723206c8947b62c86649"
Oct 30 20:11:17.566354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.566330    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j"
Oct 30 20:11:17.566487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.566345    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lx88j" event={"ID":"462e0d54-1bec-4e67-a243-72d25ffe8b1d","Type":"ContainerDied","Data":"d16aec85e309061ecaea10b9cc62ad370f39a60000c3efce1a32d13692c9e786"}
Oct 30 20:11:17.566487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.566390    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d16aec85e309061ecaea10b9cc62ad370f39a60000c3efce1a32d13692c9e786"
Oct 30 20:11:17.568321 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.568301    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r"
Oct 30 20:11:17.568435 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.568337    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2qp5r" event={"ID":"82ff46bd-4e37-4e9a-9258-7c4e3e91399a","Type":"ContainerDied","Data":"26c38e2307958135331aadd326e5213814625888ff9dbc17462b8a242e871427"}
Oct 30 20:11:17.568435 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.568374    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26c38e2307958135331aadd326e5213814625888ff9dbc17462b8a242e871427"
Oct 30 20:11:17.691964 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.691940    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:17.754137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.754096    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle\") pod \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") "
Oct 30 20:11:17.754293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.754144    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util\") pod \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") "
Oct 30 20:11:17.754293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.754170    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd\" (UniqueName: \"kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd\") pod \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\" (UID: \"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024\") "
Oct 30 20:11:17.754795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.754768    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle" pod "26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" (UID: "26a20e2b-4dc3-4a1a-9d65-6bd54f22e024"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:17.756269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.756247    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd" pod "26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" (UID: "26a20e2b-4dc3-4a1a-9d65-6bd54f22e024"). InnerVolumeSpecName "kube-api-access-5x7xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:11:17.758474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.758433    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util" pod "26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" (UID: "26a20e2b-4dc3-4a1a-9d65-6bd54f22e024"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:11:17.855743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.855670    6389 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-bundle\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:17.855743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.855694    6389 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-util\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:17.855743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:17.855704    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x7xd\" (UniqueName: \"kubernetes.io/projected/26a20e2b-4dc3-4a1a-9d65-6bd54f22e024-kube-api-access-5x7xd\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:11:18.575705 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:18.575670    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf" event={"ID":"26a20e2b-4dc3-4a1a-9d65-6bd54f22e024","Type":"ContainerDied","Data":"a4bed79a53db1d534875f3983499cd1092616a704c63a4233ffde9e329d5a853"}
Oct 30 20:11:18.576166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:18.575711    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avg5xf"
Oct 30 20:11:18.576166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:18.575720    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4bed79a53db1d534875f3983499cd1092616a704c63a4233ffde9e329d5a853"
Oct 30 20:11:25.786540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:25.786508    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:25.790560 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:11:25.790527    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:11:26.575423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:26.575393    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:29.805611 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:29.805582    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:30.038933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:30.038889    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:31.363423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.363372    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"]
Oct 30 20:11:31.364056 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364033    6389 cpu_manager.go:414] "containerMap: removing container" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364261    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364325    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364334    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364356    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364364    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364374    6389 cpu_manager.go:414] "containerMap: removing container" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364381    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364416    6389 cpu_manager.go:414] "containerMap: removing container" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364423    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364450    6389 cpu_manager.go:414] "containerMap: removing container" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364457    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="util"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364468    6389 cpu_manager.go:414] "containerMap: removing container" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364476    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364493    6389 cpu_manager.go:414] "containerMap: removing container" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364501    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364520    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364527    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364540    6389 cpu_manager.go:414] "containerMap: removing container" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364547    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="pull"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364562    6389 cpu_manager.go:414] "containerMap: removing container" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364569    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364579    6389 cpu_manager.go:414] "containerMap: removing container" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364587    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364759    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="462e0d54-1bec-4e67-a243-72d25ffe8b1d" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364778    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="b94c7047-1900-4bac-89fe-2f7431435587" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364786    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="82ff46bd-4e37-4e9a-9258-7c4e3e91399a" containerName="extract"
Oct 30 20:11:31.365841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.364795    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="26a20e2b-4dc3-4a1a-9d65-6bd54f22e024" containerName="extract"
Oct 30 20:11:31.368167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.368146    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:31.375477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.375459    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 20:11:31.377323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.377299    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"]
Oct 30 20:11:31.377671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.377643    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 20:11:31.378212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.378186    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pzfb5\""
Oct 30 20:11:31.469452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.469419    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8fts\" (UniqueName: \"kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts\") pod \"limitador-operator-controller-manager-856dd584c5-5hn8f\" (UID: \"318a03e3-3599-4e1c-9b16-04648279a196\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:31.570319 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.570279    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8fts\" (UniqueName: \"kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts\") pod \"limitador-operator-controller-manager-856dd584c5-5hn8f\" (UID: \"318a03e3-3599-4e1c-9b16-04648279a196\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:31.584520 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.584491    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8fts\" (UniqueName: \"kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts\") pod \"limitador-operator-controller-manager-856dd584c5-5hn8f\" (UID: \"318a03e3-3599-4e1c-9b16-04648279a196\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:31.681166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.681064    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:31.966611 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.966578    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"]
Oct 30 20:11:31.969109 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:11:31.969082    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318a03e3_3599_4e1c_9b16_04648279a196.slice/crio-b52c4ad0e45c076542227399f13f7de4232e6ac6c3a80f2af96d1efc3dc8eada WatchSource:0}: Error finding container b52c4ad0e45c076542227399f13f7de4232e6ac6c3a80f2af96d1efc3dc8eada: Status 404 returned error can't find the container with id b52c4ad0e45c076542227399f13f7de4232e6ac6c3a80f2af96d1efc3dc8eada
Oct 30 20:11:31.990007 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:31.989985    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:32.209704 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:32.209669    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:32.646498 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:32.646465    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f" event={"ID":"318a03e3-3599-4e1c-9b16-04648279a196","Type":"ContainerStarted","Data":"b52c4ad0e45c076542227399f13f7de4232e6ac6c3a80f2af96d1efc3dc8eada"}
Oct 30 20:11:33.057839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:33.057804    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:34.184112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:34.184077    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:34.415062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:34.415030    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:34.588248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:34.588215    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:35.187456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:35.187422    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:35.664885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:35.664839    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f" event={"ID":"318a03e3-3599-4e1c-9b16-04648279a196","Type":"ContainerStarted","Data":"6d9f7e0cb61aadc497be37a7eaf5223022a23a589131fe8b1323724abfe89187"}
Oct 30 20:11:35.665087 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:35.665035    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:35.669246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:35.669223    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:35.705951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:35.705910    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f" podStartSLOduration=1.329863443 podStartE2EDuration="4.705894783s" totalImagesPullingTime="3.37603134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:11:35.705319537 +0000 UTC m=+415.904892464" watchObservedRunningTime="2026-10-30 20:11:35.705894783 +0000 UTC m=+415.905467710"
Oct 30 20:11:35.792575 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:35.792546    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:39.818839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:39.818807    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:39.933462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:39.933437    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:40.388927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:40.388895    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:40.495573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:40.495545    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:42.811377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:42.809891    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:42.912628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:42.912600    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:45.388204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:45.388171    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:45.490420 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:45.490381    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:46.500758 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.500728    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"]
Oct 30 20:11:46.505334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.505312    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.514642 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.514613    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 20:11:46.514788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.514714    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 20:11:46.514875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.514854    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6qhrp\""
Oct 30 20:11:46.515500 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.515473    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"]
Oct 30 20:11:46.518257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.518233    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:46.614186 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.614154    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ff7\" (UniqueName: \"kubernetes.io/projected/50eaa387-2837-4f47-a710-139ad81cccf1-kube-api-access-w9ff7\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.614333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.614190    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/50eaa387-2837-4f47-a710-139ad81cccf1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.614333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.614210    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/50eaa387-2837-4f47-a710-139ad81cccf1-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.617448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.617425    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:46.672850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.672828    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:46.715338 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.715312    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ff7\" (UniqueName: \"kubernetes.io/projected/50eaa387-2837-4f47-a710-139ad81cccf1-kube-api-access-w9ff7\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.715489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.715351    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/50eaa387-2837-4f47-a710-139ad81cccf1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.715489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.715376    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/50eaa387-2837-4f47-a710-139ad81cccf1-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.716227 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.716191    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/50eaa387-2837-4f47-a710-139ad81cccf1-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.717746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.717726    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/50eaa387-2837-4f47-a710-139ad81cccf1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.728623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.728601    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ff7\" (UniqueName: \"kubernetes.io/projected/50eaa387-2837-4f47-a710-139ad81cccf1-kube-api-access-w9ff7\") pod \"kuadrant-console-plugin-66c4fb575d-x6hnt\" (UID: \"50eaa387-2837-4f47-a710-139ad81cccf1\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:46.817200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:46.817109    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"
Oct 30 20:11:47.121319 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:47.121295    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt"]
Oct 30 20:11:47.123793 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:11:47.123751    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50eaa387_2837_4f47_a710_139ad81cccf1.slice/crio-458b116bce3c9f2bad03deaad3639bdef0a53394683648f231944787a77d9ab3 WatchSource:0}: Error finding container 458b116bce3c9f2bad03deaad3639bdef0a53394683648f231944787a77d9ab3: Status 404 returned error can't find the container with id 458b116bce3c9f2bad03deaad3639bdef0a53394683648f231944787a77d9ab3
Oct 30 20:11:47.610146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:47.610087    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:47.714773 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:47.714741    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:47.721793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:47.721758    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt" event={"ID":"50eaa387-2837-4f47-a710-139ad81cccf1","Type":"ContainerStarted","Data":"458b116bce3c9f2bad03deaad3639bdef0a53394683648f231944787a77d9ab3"}
Oct 30 20:11:51.819065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:51.819030    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:51.927170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:51.927107    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:52.754167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:52.754134    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt" event={"ID":"50eaa387-2837-4f47-a710-139ad81cccf1","Type":"ContainerStarted","Data":"16b68464074fd135935ec26610e3069f3c384d17214eddca76da516da4355e3f"}
Oct 30 20:11:52.773828 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:52.773781    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-x6hnt" podStartSLOduration=1.4828681910000001 podStartE2EDuration="6.773767425s" totalImagesPullingTime="5.290899234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:11:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:11:52.772478194 +0000 UTC m=+432.972051309" watchObservedRunningTime="2026-10-30 20:11:52.773767425 +0000 UTC m=+432.973340352"
Oct 30 20:11:55.792295 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:11:55.792264    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:11:58.837478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:58.837447    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:58.942874 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:58.942841    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:11:59.562871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.562830    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"]
Oct 30 20:11:59.563452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.563177    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f" podUID="318a03e3-3599-4e1c-9b16-04648279a196" containerName="manager" containerID="cri-o://6d9f7e0cb61aadc497be37a7eaf5223022a23a589131fe8b1323724abfe89187" gracePeriod=2
Oct 30 20:11:59.566792 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:11:59.566736    6389 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-5hn8f\" is forbidden: User \"system:node:ip-10-0-129-14.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-14.us-east-2.compute.internal' and this object" podUID="318a03e3-3599-4e1c-9b16-04648279a196" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:59.568600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.568576    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"]
Oct 30 20:11:59.593222 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.593191    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"]
Oct 30 20:11:59.593685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.593665    6389 cpu_manager.go:414] "containerMap: removing container" podUID="318a03e3-3599-4e1c-9b16-04648279a196" containerName="manager"
Oct 30 20:11:59.593736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.593690    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="318a03e3-3599-4e1c-9b16-04648279a196" containerName="manager"
Oct 30 20:11:59.593879 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.593865    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="318a03e3-3599-4e1c-9b16-04648279a196" containerName="manager"
Oct 30 20:11:59.597211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.597193    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:11:59.600005 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:11:59.599978    6389 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-5hn8f\" is forbidden: User \"system:node:ip-10-0-129-14.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-14.us-east-2.compute.internal' and this object" podUID="318a03e3-3599-4e1c-9b16-04648279a196" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:59.606398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.606374    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"]
Oct 30 20:11:59.631995 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.631968    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxx92\" (UniqueName: \"kubernetes.io/projected/8d8ad07a-926f-47e2-a949-3dfcac0aae97-kube-api-access-zxx92\") pod \"limitador-operator-controller-manager-856dd584c5-lspfl\" (UID: \"8d8ad07a-926f-47e2-a949-3dfcac0aae97\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:11:59.732771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.732744    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxx92\" (UniqueName: \"kubernetes.io/projected/8d8ad07a-926f-47e2-a949-3dfcac0aae97-kube-api-access-zxx92\") pod \"limitador-operator-controller-manager-856dd584c5-lspfl\" (UID: \"8d8ad07a-926f-47e2-a949-3dfcac0aae97\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:11:59.749691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.749661    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxx92\" (UniqueName: \"kubernetes.io/projected/8d8ad07a-926f-47e2-a949-3dfcac0aae97-kube-api-access-zxx92\") pod \"limitador-operator-controller-manager-856dd584c5-lspfl\" (UID: \"8d8ad07a-926f-47e2-a949-3dfcac0aae97\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:11:59.797758 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.797728    6389 generic.go:356] "Generic (PLEG): container finished" podID="318a03e3-3599-4e1c-9b16-04648279a196" containerID="6d9f7e0cb61aadc497be37a7eaf5223022a23a589131fe8b1323724abfe89187" exitCode=0
Oct 30 20:11:59.797882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.797818    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b52c4ad0e45c076542227399f13f7de4232e6ac6c3a80f2af96d1efc3dc8eada"
Oct 30 20:11:59.802779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.802756    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:59.805295 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:11:59.805272    6389 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-5hn8f\" is forbidden: User \"system:node:ip-10-0-129-14.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-14.us-east-2.compute.internal' and this object" podUID="318a03e3-3599-4e1c-9b16-04648279a196" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:59.833989 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.833932    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts\" (UniqueName: \"kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts\") pod \"318a03e3-3599-4e1c-9b16-04648279a196\" (UID: \"318a03e3-3599-4e1c-9b16-04648279a196\") "
Oct 30 20:11:59.835978 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.835954    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts" pod "318a03e3-3599-4e1c-9b16-04648279a196" (UID: "318a03e3-3599-4e1c-9b16-04648279a196"). InnerVolumeSpecName "kube-api-access-l8fts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:11:59.912543 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:11:59.912505    6389 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-5hn8f\" is forbidden: User \"system:node:ip-10-0-129-14.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-14.us-east-2.compute.internal' and this object" podUID="318a03e3-3599-4e1c-9b16-04648279a196" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:11:59.912920 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.912900    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="318a03e3-3599-4e1c-9b16-04648279a196" path="/var/lib/kubelet/pods/318a03e3-3599-4e1c-9b16-04648279a196/volumes"
Oct 30 20:11:59.929762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.929739    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:11:59.934699 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:11:59.934679    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8fts\" (UniqueName: \"kubernetes.io/projected/318a03e3-3599-4e1c-9b16-04648279a196-kube-api-access-l8fts\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:00.212380 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:00.212355    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"]
Oct 30 20:12:00.215037 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:12:00.215012    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8ad07a_926f_47e2_a949_3dfcac0aae97.slice/crio-92d661054136bffc631c854669441b05f7e481d188c4e3dbb6049c497810896d WatchSource:0}: Error finding container 92d661054136bffc631c854669441b05f7e481d188c4e3dbb6049c497810896d: Status 404 returned error can't find the container with id 92d661054136bffc631c854669441b05f7e481d188c4e3dbb6049c497810896d
Oct 30 20:12:00.811701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:00.811655    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl" event={"ID":"8d8ad07a-926f-47e2-a949-3dfcac0aae97","Type":"ContainerStarted","Data":"55f10a04ce4fd95f5b6dfa3266d4ef82d4f41c3052b70fd8ed3a5eb6813bcf94"}
Oct 30 20:12:00.811878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:00.811711    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-5hn8f"
Oct 30 20:12:00.811878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:00.811729    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl" event={"ID":"8d8ad07a-926f-47e2-a949-3dfcac0aae97","Type":"ContainerStarted","Data":"92d661054136bffc631c854669441b05f7e481d188c4e3dbb6049c497810896d"}
Oct 30 20:12:00.811878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:00.811783    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:12:00.834234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:00.834183    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl" podStartSLOduration=1.834168507 podStartE2EDuration="1.834168507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:11:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:12:00.833159246 +0000 UTC m=+441.032732176" watchObservedRunningTime="2026-10-30 20:12:00.834168507 +0000 UTC m=+441.033741435"
Oct 30 20:12:02.442500 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:02.442466    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:02.548977 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:02.548936    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:05.439821 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:05.439785    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:05.543780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:05.543746    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:09.024814 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:09.024772    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:11.077591 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:11.077558    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:11.184534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:11.184496    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:11.818484 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:11.818458    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-lspfl"
Oct 30 20:12:25.790954 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:12:25.790918    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:12:26.617489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:26.617443    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:26.719446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:26.719413    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:31.432609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:31.432574    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:31.547590 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:31.547558    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:33.082341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:33.082297    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:33.186626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:33.186590    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:42.043490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.043451    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:42.067530 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.067492    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:42.067674 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.067616    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.074935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.074912    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 20:12:42.078871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.078849    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:42.114729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.114700    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file\") pod \"limitador-limitador-74dfcfb894-852t5\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.114885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.114814    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh\") pod \"limitador-limitador-74dfcfb894-852t5\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.144248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.144221    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:42.191801 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.191771    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:42.215988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.215957    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh\") pod \"limitador-limitador-74dfcfb894-852t5\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.216145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.215994    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file\") pod \"limitador-limitador-74dfcfb894-852t5\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.216579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.216562    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file\") pod \"limitador-limitador-74dfcfb894-852t5\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.229233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.229208    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh\") pod \"limitador-limitador-74dfcfb894-852t5\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.293735 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.293670    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:42.380159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.380110    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:42.657518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.657412    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:42.659616 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:12:42.659587    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ba31e6_eda0_4a23_86ae_d4005854af48.slice/crio-c33c08fda842936d118a3add7046070d7c8b9c0590e047f9c4c94a35c8ab2409 WatchSource:0}: Error finding container c33c08fda842936d118a3add7046070d7c8b9c0590e047f9c4c94a35c8ab2409: Status 404 returned error can't find the container with id c33c08fda842936d118a3add7046070d7c8b9c0590e047f9c4c94a35c8ab2409
Oct 30 20:12:42.920243 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.920163    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5p25"]
Oct 30 20:12:42.924894 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.924866    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:42.933259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.933236    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x29mr\""
Oct 30 20:12:42.933551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.933527    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5p25"]
Oct 30 20:12:42.984870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:42.984840    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:43.014915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.014885    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5" event={"ID":"d0ba31e6-eda0-4a23-86ae-d4005854af48","Type":"ContainerStarted","Data":"c33c08fda842936d118a3add7046070d7c8b9c0590e047f9c4c94a35c8ab2409"}
Oct 30 20:12:43.022382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.022355    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q86cp\" (UniqueName: \"kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp\") pod \"authorino-57f7f8659f-x5p25\" (UID: \"70b06196-a0e6-4829-b9c6-ecd46464f0d7\") " pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:43.097031 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.096998    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:12:43.123768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.123737    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q86cp\" (UniqueName: \"kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp\") pod \"authorino-57f7f8659f-x5p25\" (UID: \"70b06196-a0e6-4829-b9c6-ecd46464f0d7\") " pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:43.137067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.137037    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q86cp\" (UniqueName: \"kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp\") pod \"authorino-57f7f8659f-x5p25\" (UID: \"70b06196-a0e6-4829-b9c6-ecd46464f0d7\") " pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:43.237778 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.237747    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:43.508065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:43.507975    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5p25"]
Oct 30 20:12:43.511575 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:12:43.511549    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70b06196_a0e6_4829_b9c6_ecd46464f0d7.slice/crio-06eb8c96b1f4f2dcb25664a28f90664a268331d2776e1e9392defb3cd464b7a7 WatchSource:0}: Error finding container 06eb8c96b1f4f2dcb25664a28f90664a268331d2776e1e9392defb3cd464b7a7: Status 404 returned error can't find the container with id 06eb8c96b1f4f2dcb25664a28f90664a268331d2776e1e9392defb3cd464b7a7
Oct 30 20:12:44.020249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:44.020209    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5p25" event={"ID":"70b06196-a0e6-4829-b9c6-ecd46464f0d7","Type":"ContainerStarted","Data":"06eb8c96b1f4f2dcb25664a28f90664a268331d2776e1e9392defb3cd464b7a7"}
Oct 30 20:12:45.026790 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:45.026756    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5" event={"ID":"d0ba31e6-eda0-4a23-86ae-d4005854af48","Type":"ContainerStarted","Data":"3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea"}
Oct 30 20:12:45.027206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:45.026915    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:45.047978 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:45.047929    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5" podStartSLOduration=0.772096292 podStartE2EDuration="3.047914798s" totalImagesPullingTime="2.275818506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:12:45.046581591 +0000 UTC m=+485.246154553" watchObservedRunningTime="2026-10-30 20:12:45.047914798 +0000 UTC m=+485.247487726"
Oct 30 20:12:46.033385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:46.033329    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5p25" event={"ID":"70b06196-a0e6-4829-b9c6-ecd46464f0d7","Type":"ContainerStarted","Data":"a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d"}
Oct 30 20:12:46.061858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:46.061813    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-x5p25" podStartSLOduration=2.033514142 podStartE2EDuration="4.061798722s" totalImagesPullingTime="2.02828458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:12:46.060104342 +0000 UTC m=+486.259677281" watchObservedRunningTime="2026-10-30 20:12:46.061798722 +0000 UTC m=+486.261371649"
Oct 30 20:12:46.073750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:46.073724    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5p25"]
Oct 30 20:12:48.042645 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:48.042599    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-x5p25" podUID="70b06196-a0e6-4829-b9c6-ecd46464f0d7" containerName="authorino" containerID="cri-o://a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d" gracePeriod=30
Oct 30 20:12:48.285626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:48.285599    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:48.369561 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:48.369474    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp\" (UniqueName: \"kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp\") pod \"70b06196-a0e6-4829-b9c6-ecd46464f0d7\" (UID: \"70b06196-a0e6-4829-b9c6-ecd46464f0d7\") "
Oct 30 20:12:48.371519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:48.371494    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp" pod "70b06196-a0e6-4829-b9c6-ecd46464f0d7" (UID: "70b06196-a0e6-4829-b9c6-ecd46464f0d7"). InnerVolumeSpecName "kube-api-access-q86cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:12:48.470105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:48.470059    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q86cp\" (UniqueName: \"kubernetes.io/projected/70b06196-a0e6-4829-b9c6-ecd46464f0d7-kube-api-access-q86cp\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:49.048486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.048454    6389 generic.go:356] "Generic (PLEG): container finished" podID="70b06196-a0e6-4829-b9c6-ecd46464f0d7" containerID="a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d" exitCode=0
Oct 30 20:12:49.048912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.048498    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5p25"
Oct 30 20:12:49.048912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.048545    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5p25" event={"ID":"70b06196-a0e6-4829-b9c6-ecd46464f0d7","Type":"ContainerDied","Data":"a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d"}
Oct 30 20:12:49.048912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.048591    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5p25" event={"ID":"70b06196-a0e6-4829-b9c6-ecd46464f0d7","Type":"ContainerDied","Data":"06eb8c96b1f4f2dcb25664a28f90664a268331d2776e1e9392defb3cd464b7a7"}
Oct 30 20:12:49.048912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.048611    6389 scope.go:122] "RemoveContainer" containerID="a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d"
Oct 30 20:12:49.057605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.057586    6389 scope.go:122] "RemoveContainer" containerID="a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d"
Oct 30 20:12:49.057870 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:12:49.057854    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d\": container with ID starting with a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d not found: ID does not exist" containerID="a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d"
Oct 30 20:12:49.057919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.057881    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d"} err="failed to get container status \"a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d\": rpc error: code = NotFound desc = could not find container \"a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d\": container with ID starting with a1f72abab179253046171e5e1c8d1f76b95b0fa8c28ae82a54c7fdd30bd5675d not found: ID does not exist"
Oct 30 20:12:49.071590 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.071564    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5p25"]
Oct 30 20:12:49.074249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.074227    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5p25"]
Oct 30 20:12:49.912897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:49.912867    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70b06196-a0e6-4829-b9c6-ecd46464f0d7" path="/var/lib/kubelet/pods/70b06196-a0e6-4829-b9c6-ecd46464f0d7/volumes"
Oct 30 20:12:55.790533 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:12:55.790495    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:12:56.034426 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.034400    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:56.088661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.088589    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:56.088828 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.088784    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5" podUID="d0ba31e6-eda0-4a23-86ae-d4005854af48" containerName="limitador" containerID="cri-o://3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea" gracePeriod=30
Oct 30 20:12:56.624971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.624947    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:56.734230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.734206    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file\" (UniqueName: \"kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file\") pod \"d0ba31e6-eda0-4a23-86ae-d4005854af48\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") "
Oct 30 20:12:56.734389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.734248    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh\") pod \"d0ba31e6-eda0-4a23-86ae-d4005854af48\" (UID: \"d0ba31e6-eda0-4a23-86ae-d4005854af48\") "
Oct 30 20:12:56.734612 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.734583    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file" pod "d0ba31e6-eda0-4a23-86ae-d4005854af48" (UID: "d0ba31e6-eda0-4a23-86ae-d4005854af48"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:12:56.736330 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.736303    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh" pod "d0ba31e6-eda0-4a23-86ae-d4005854af48" (UID: "d0ba31e6-eda0-4a23-86ae-d4005854af48"). InnerVolumeSpecName "kube-api-access-hc2qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:12:56.834836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.834814    6389 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d0ba31e6-eda0-4a23-86ae-d4005854af48-config-file\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:56.834836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:56.834835    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/d0ba31e6-eda0-4a23-86ae-d4005854af48-kube-api-access-hc2qh\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:57.088880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.088792    6389 generic.go:356] "Generic (PLEG): container finished" podID="d0ba31e6-eda0-4a23-86ae-d4005854af48" containerID="3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea" exitCode=0
Oct 30 20:12:57.088880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.088849    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5"
Oct 30 20:12:57.089091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.088854    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5" event={"ID":"d0ba31e6-eda0-4a23-86ae-d4005854af48","Type":"ContainerDied","Data":"3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea"}
Oct 30 20:12:57.089091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.088968    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-852t5" event={"ID":"d0ba31e6-eda0-4a23-86ae-d4005854af48","Type":"ContainerDied","Data":"c33c08fda842936d118a3add7046070d7c8b9c0590e047f9c4c94a35c8ab2409"}
Oct 30 20:12:57.089091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.088992    6389 scope.go:122] "RemoveContainer" containerID="3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea"
Oct 30 20:12:57.097447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.097427    6389 scope.go:122] "RemoveContainer" containerID="3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea"
Oct 30 20:12:57.097691 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:12:57.097669    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea\": container with ID starting with 3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea not found: ID does not exist" containerID="3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea"
Oct 30 20:12:57.097736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.097701    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea"} err="failed to get container status \"3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea\": rpc error: code = NotFound desc = could not find container \"3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea\": container with ID starting with 3862653683a4a050945ca70f5b81a285bc502a31a6c64fbddfd94a7beaf598ea not found: ID does not exist"
Oct 30 20:12:57.112368 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.112347    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:57.118627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.118608    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-852t5"]
Oct 30 20:12:57.912135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:12:57.912085    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0ba31e6-eda0-4a23-86ae-d4005854af48" path="/var/lib/kubelet/pods/d0ba31e6-eda0-4a23-86ae-d4005854af48/volumes"
Oct 30 20:13:00.751243 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:00.751212    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:00.854509 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:00.854478    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:01.468204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468170    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"]
Oct 30 20:13:01.468545 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468533    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d0ba31e6-eda0-4a23-86ae-d4005854af48" containerName="limitador"
Oct 30 20:13:01.468598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468547    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ba31e6-eda0-4a23-86ae-d4005854af48" containerName="limitador"
Oct 30 20:13:01.468598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468565    6389 cpu_manager.go:414] "containerMap: removing container" podUID="70b06196-a0e6-4829-b9c6-ecd46464f0d7" containerName="authorino"
Oct 30 20:13:01.468598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468570    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70b06196-a0e6-4829-b9c6-ecd46464f0d7" containerName="authorino"
Oct 30 20:13:01.468686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468627    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ba31e6-eda0-4a23-86ae-d4005854af48" containerName="limitador"
Oct 30 20:13:01.468686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.468638    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="70b06196-a0e6-4829-b9c6-ecd46464f0d7" containerName="authorino"
Oct 30 20:13:01.473035 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.473018    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.480804 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.480778    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"]
Oct 30 20:13:01.481184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.481168    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 20:13:01.481305 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.481219    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 20:13:01.481372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.481347    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-jhqp2\""
Oct 30 20:13:01.481417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.481371    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 20:13:01.554885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.554853    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:01.570064 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.570040    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d974ae8a-b8fd-4583-ab47-a3e6d408bd53-data\") pod \"postgres-7cdbf6dc44-snrhr\" (UID: \"d974ae8a-b8fd-4583-ab47-a3e6d408bd53\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.570162 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.570135    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnp74\" (UniqueName: \"kubernetes.io/projected/d974ae8a-b8fd-4583-ab47-a3e6d408bd53-kube-api-access-fnp74\") pod \"postgres-7cdbf6dc44-snrhr\" (UID: \"d974ae8a-b8fd-4583-ab47-a3e6d408bd53\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.671400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.671355    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d974ae8a-b8fd-4583-ab47-a3e6d408bd53-data\") pod \"postgres-7cdbf6dc44-snrhr\" (UID: \"d974ae8a-b8fd-4583-ab47-a3e6d408bd53\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.671543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.671410    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnp74\" (UniqueName: \"kubernetes.io/projected/d974ae8a-b8fd-4583-ab47-a3e6d408bd53-kube-api-access-fnp74\") pod \"postgres-7cdbf6dc44-snrhr\" (UID: \"d974ae8a-b8fd-4583-ab47-a3e6d408bd53\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.671751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.671730    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d974ae8a-b8fd-4583-ab47-a3e6d408bd53-data\") pod \"postgres-7cdbf6dc44-snrhr\" (UID: \"d974ae8a-b8fd-4583-ab47-a3e6d408bd53\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.687431 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.687404    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnp74\" (UniqueName: \"kubernetes.io/projected/d974ae8a-b8fd-4583-ab47-a3e6d408bd53-kube-api-access-fnp74\") pod \"postgres-7cdbf6dc44-snrhr\" (UID: \"d974ae8a-b8fd-4583-ab47-a3e6d408bd53\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:01.787749 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:01.787675    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:02.061976 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:02.061929    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"]
Oct 30 20:13:02.064256 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:13:02.064227    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd974ae8a_b8fd_4583_ab47_a3e6d408bd53.slice/crio-c09fdac5eb098d5388776d428a2d61690e1a9c52441813aa9995abb09c489072 WatchSource:0}: Error finding container c09fdac5eb098d5388776d428a2d61690e1a9c52441813aa9995abb09c489072: Status 404 returned error can't find the container with id c09fdac5eb098d5388776d428a2d61690e1a9c52441813aa9995abb09c489072
Oct 30 20:13:02.065570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:02.065554    6389 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:13:02.120241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:02.120209    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr" event={"ID":"d974ae8a-b8fd-4583-ab47-a3e6d408bd53","Type":"ContainerStarted","Data":"c09fdac5eb098d5388776d428a2d61690e1a9c52441813aa9995abb09c489072"}
Oct 30 20:13:08.159214 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:08.159169    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr" event={"ID":"d974ae8a-b8fd-4583-ab47-a3e6d408bd53","Type":"ContainerStarted","Data":"9e2a5e92f08c7dda5fb0d82ec70264ec6050e8845db20f04dbd371a9a24d8385"}
Oct 30 20:13:08.159733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:08.159241    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:08.180772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:08.180696    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr" podStartSLOduration=1.730791806 podStartE2EDuration="7.180681877s" totalImagesPullingTime="5.449890071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:13:08.179396294 +0000 UTC m=+508.378969223" watchObservedRunningTime="2026-10-30 20:13:08.180681877 +0000 UTC m=+508.380254805"
Oct 30 20:13:14.193730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.193702    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-snrhr"
Oct 30 20:13:14.659780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.659745    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:14.824868 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.824841    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:14.968662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.968633    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd76875b5-wggrj"]
Oct 30 20:13:14.974694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.974678    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:14.981453 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.981382    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x29mr\""
Oct 30 20:13:14.981544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.981465    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 20:13:14.982773 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:14.982747    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-wggrj"]
Oct 30 20:13:15.065109 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.065076    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-wggrj"]
Oct 30 20:13:15.065300 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:13:15.065281    6389 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-c9d6g tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6dd76875b5-wggrj" podUID="96dfd065-aff7-41eb-ae1a-9e5b440eb37b"
Oct 30 20:13:15.088750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.088722    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert\") pod \"authorino-6dd76875b5-wggrj\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") " pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.088896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.088791    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9d6g\" (UniqueName: \"kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g\") pod \"authorino-6dd76875b5-wggrj\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") " pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.131969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.131830    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-7jp89"]
Oct 30 20:13:15.136401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.136385    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.143957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.143934    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-7jp89"]
Oct 30 20:13:15.189558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.189536    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert\") pod \"authorino-6dd76875b5-wggrj\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") " pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.189685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.189616    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9d6g\" (UniqueName: \"kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g\") pod \"authorino-6dd76875b5-wggrj\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") " pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.192258 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.192237    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert\") pod \"authorino-6dd76875b5-wggrj\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") " pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.192708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.192690    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.204328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.204308    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9d6g\" (UniqueName: \"kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g\") pod \"authorino-6dd76875b5-wggrj\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") " pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.206328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.206314    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:15.290961 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.290903    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g\" (UniqueName: \"kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g\") pod \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") "
Oct 30 20:13:15.290961 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.290941    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert\" (UniqueName: \"kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert\") pod \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\" (UID: \"96dfd065-aff7-41eb-ae1a-9e5b440eb37b\") "
Oct 30 20:13:15.291097 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.291030    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpsrn\" (UniqueName: \"kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn\") pod \"authorino-6889cbd8c4-7jp89\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") " pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.291213 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.291195    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert\") pod \"authorino-6889cbd8c4-7jp89\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") " pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.292926 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.292906    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g" pod "96dfd065-aff7-41eb-ae1a-9e5b440eb37b" (UID: "96dfd065-aff7-41eb-ae1a-9e5b440eb37b"). InnerVolumeSpecName "kube-api-access-c9d6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:13:15.292997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.292922    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert" pod "96dfd065-aff7-41eb-ae1a-9e5b440eb37b" (UID: "96dfd065-aff7-41eb-ae1a-9e5b440eb37b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:13:15.350588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.350560    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-7jp89"]
Oct 30 20:13:15.350744 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:13:15.350726    6389 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cpsrn tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-7jp89" podUID="f0e0a635-9411-4d41-a064-e3a6a9b35fe3"
Oct 30 20:13:15.379564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.379536    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c8cf5b685-hv8q4"]
Oct 30 20:13:15.383503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.383485    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.391774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.391753    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c8cf5b685-hv8q4"]
Oct 30 20:13:15.392026 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.392007    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpsrn\" (UniqueName: \"kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn\") pod \"authorino-6889cbd8c4-7jp89\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") " pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.392198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.392177    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert\") pod \"authorino-6889cbd8c4-7jp89\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") " pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.392276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.392233    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9d6g\" (UniqueName: \"kubernetes.io/projected/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-kube-api-access-c9d6g\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:13:15.392276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.392249    6389 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96dfd065-aff7-41eb-ae1a-9e5b440eb37b-tls-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:13:15.395289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.395267    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert\") pod \"authorino-6889cbd8c4-7jp89\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") " pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.405301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.405280    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpsrn\" (UniqueName: \"kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn\") pod \"authorino-6889cbd8c4-7jp89\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") " pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:15.492822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.492796    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r89lq\" (UniqueName: \"kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq\") pod \"authorino-7c8cf5b685-hv8q4\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") " pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.492926 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.492852    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert\") pod \"authorino-7c8cf5b685-hv8q4\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") " pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.593731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.593650    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r89lq\" (UniqueName: \"kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq\") pod \"authorino-7c8cf5b685-hv8q4\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") " pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.593731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.593707    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert\") pod \"authorino-7c8cf5b685-hv8q4\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") " pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.596182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.596159    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert\") pod \"authorino-7c8cf5b685-hv8q4\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") " pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.606939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.606917    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r89lq\" (UniqueName: \"kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq\") pod \"authorino-7c8cf5b685-hv8q4\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") " pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.696885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.696857    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:13:15.967935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:15.967911    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c8cf5b685-hv8q4"]
Oct 30 20:13:15.970038 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:13:15.970011    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca401ae1_d4da_4055_b3f9_46a532696d80.slice/crio-339110cad589313f34cfb9b111642a8c0a2b2336ffcfa1431e7ec7ae8f0fcc97 WatchSource:0}: Error finding container 339110cad589313f34cfb9b111642a8c0a2b2336ffcfa1431e7ec7ae8f0fcc97: Status 404 returned error can't find the container with id 339110cad589313f34cfb9b111642a8c0a2b2336ffcfa1431e7ec7ae8f0fcc97
Oct 30 20:13:16.198293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.198212    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:16.198293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.198242    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-wggrj"
Oct 30 20:13:16.198497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.198216    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4" event={"ID":"ca401ae1-d4da-4055-b3f9-46a532696d80","Type":"ContainerStarted","Data":"339110cad589313f34cfb9b111642a8c0a2b2336ffcfa1431e7ec7ae8f0fcc97"}
Oct 30 20:13:16.212646 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.212627    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:16.255062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.255035    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-wggrj"]
Oct 30 20:13:16.264718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.264690    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-wggrj"]
Oct 30 20:13:16.300230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.300205    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert\" (UniqueName: \"kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert\") pod \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") "
Oct 30 20:13:16.300324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.300254    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn\" (UniqueName: \"kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn\") pod \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\" (UID: \"f0e0a635-9411-4d41-a064-e3a6a9b35fe3\") "
Oct 30 20:13:16.302226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.302198    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert" pod "f0e0a635-9411-4d41-a064-e3a6a9b35fe3" (UID: "f0e0a635-9411-4d41-a064-e3a6a9b35fe3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:13:16.302226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.302218    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn" pod "f0e0a635-9411-4d41-a064-e3a6a9b35fe3" (UID: "f0e0a635-9411-4d41-a064-e3a6a9b35fe3"). InnerVolumeSpecName "kube-api-access-cpsrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:13:16.401757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.401730    6389 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-tls-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:13:16.401757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:16.401755    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpsrn\" (UniqueName: \"kubernetes.io/projected/f0e0a635-9411-4d41-a064-e3a6a9b35fe3-kube-api-access-cpsrn\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:13:17.204714 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.204623    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4" event={"ID":"ca401ae1-d4da-4055-b3f9-46a532696d80","Type":"ContainerStarted","Data":"3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994"}
Oct 30 20:13:17.204863 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.204715    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-7jp89"
Oct 30 20:13:17.223163 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.223110    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4" podStartSLOduration=1.449672499 podStartE2EDuration="2.223095094s" totalImagesPullingTime="773.422595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:13:17.222370653 +0000 UTC m=+517.421943582" watchObservedRunningTime="2026-10-30 20:13:17.223095094 +0000 UTC m=+517.422668021"
Oct 30 20:13:17.256287 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.256257    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-7jp89"]
Oct 30 20:13:17.261047 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.261020    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-7jp89"]
Oct 30 20:13:17.913220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.913180    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96dfd065-aff7-41eb-ae1a-9e5b440eb37b" path="/var/lib/kubelet/pods/96dfd065-aff7-41eb-ae1a-9e5b440eb37b/volumes"
Oct 30 20:13:17.913515 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:17.913497    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0e0a635-9411-4d41-a064-e3a6a9b35fe3" path="/var/lib/kubelet/pods/f0e0a635-9411-4d41-a064-e3a6a9b35fe3/volumes"
Oct 30 20:13:24.766385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:24.766344    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:24.874782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:24.874733    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:24.979407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:24.979376    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:25.084667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:25.084555    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:25.790449 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:13:25.790417    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:13:28.614788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:28.614749    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:28.718510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:28.718467    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:31.962004 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:31.961970    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:32.090506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:32.090475    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:33.189999 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:33.189950    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:41.995256 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:13:41.995223    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:13:55.790672 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:13:55.790579    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:14:00.632960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:00.632854    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:01.415877 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:01.415844    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:01.964080 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:01.964043    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:02.065553 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:02.065518    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:02.177307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:02.177280    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:11.781593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.781558    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"]
Oct 30 20:14:11.785019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.785000    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:11.794834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.794809    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"]
Oct 30 20:14:11.886253 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.886219    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9a99f11-b317-40da-8415-cced163bede5-cert\") pod \"llmisvc-controller-manager-6466c554d8-jgvzc\" (UID: \"b9a99f11-b317-40da-8415-cced163bede5\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:11.886431 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.886340    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25s7n\" (UniqueName: \"kubernetes.io/projected/b9a99f11-b317-40da-8415-cced163bede5-kube-api-access-25s7n\") pod \"llmisvc-controller-manager-6466c554d8-jgvzc\" (UID: \"b9a99f11-b317-40da-8415-cced163bede5\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:11.987690 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.987652    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25s7n\" (UniqueName: \"kubernetes.io/projected/b9a99f11-b317-40da-8415-cced163bede5-kube-api-access-25s7n\") pod \"llmisvc-controller-manager-6466c554d8-jgvzc\" (UID: \"b9a99f11-b317-40da-8415-cced163bede5\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:11.987876 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.987719    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9a99f11-b317-40da-8415-cced163bede5-cert\") pod \"llmisvc-controller-manager-6466c554d8-jgvzc\" (UID: \"b9a99f11-b317-40da-8415-cced163bede5\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:11.990094 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:11.990069    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9a99f11-b317-40da-8415-cced163bede5-cert\") pod \"llmisvc-controller-manager-6466c554d8-jgvzc\" (UID: \"b9a99f11-b317-40da-8415-cced163bede5\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:12.000895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:12.000869    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25s7n\" (UniqueName: \"kubernetes.io/projected/b9a99f11-b317-40da-8415-cced163bede5-kube-api-access-25s7n\") pod \"llmisvc-controller-manager-6466c554d8-jgvzc\" (UID: \"b9a99f11-b317-40da-8415-cced163bede5\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:12.099223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:12.099140    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:12.371997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:12.371972    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"]
Oct 30 20:14:12.374580 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:14:12.374552    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a99f11_b317_40da_8415_cced163bede5.slice/crio-bad3993c295c1f62401a8422bc3673eefa7f99e8d4faa6df2def6934e0d1ef9a WatchSource:0}: Error finding container bad3993c295c1f62401a8422bc3673eefa7f99e8d4faa6df2def6934e0d1ef9a: Status 404 returned error can't find the container with id bad3993c295c1f62401a8422bc3673eefa7f99e8d4faa6df2def6934e0d1ef9a
Oct 30 20:14:12.472100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:12.472069    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc" event={"ID":"b9a99f11-b317-40da-8415-cced163bede5","Type":"ContainerStarted","Data":"bad3993c295c1f62401a8422bc3673eefa7f99e8d4faa6df2def6934e0d1ef9a"}
Oct 30 20:14:13.478032 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:13.477993    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc" event={"ID":"b9a99f11-b317-40da-8415-cced163bede5","Type":"ContainerStarted","Data":"f61f0479c8d34190cf070b3538d82e35617bcfa875e4d03f327bd0a0ea902a2c"}
Oct 30 20:14:13.478448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:13.478062    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:13.499732 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:13.499685    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc" podStartSLOduration=2.086489068 podStartE2EDuration="2.499671532s" totalImagesPullingTime="413.182464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:14:13.498514105 +0000 UTC m=+573.698087033" watchObservedRunningTime="2026-10-30 20:14:13.499671532 +0000 UTC m=+573.699244460"
Oct 30 20:14:25.790469 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:14:25.790434    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:14:31.328025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:31.327991    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:33.194152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:33.194089    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:39.981914 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:39.981890    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:14:39.982373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:39.982046    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:14:44.484656 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.484622    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jgvzc"
Oct 30 20:14:44.536284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.536252    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"]
Oct 30 20:14:44.536490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.536463    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc" podUID="6e852c03-a5bd-44b5-8792-ef2096c89a3f" containerName="manager" containerID="cri-o://df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405" gracePeriod=30
Oct 30 20:14:44.779074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.779052    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:14:44.867764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.867734    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert\" (UniqueName: \"kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert\") pod \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") "
Oct 30 20:14:44.867969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.867829    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr\" (UniqueName: \"kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr\") pod \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\" (UID: \"6e852c03-a5bd-44b5-8792-ef2096c89a3f\") "
Oct 30 20:14:44.869939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.869912    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr" pod "6e852c03-a5bd-44b5-8792-ef2096c89a3f" (UID: "6e852c03-a5bd-44b5-8792-ef2096c89a3f"). InnerVolumeSpecName "kube-api-access-h2lzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:44.870065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.869952    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert" pod "6e852c03-a5bd-44b5-8792-ef2096c89a3f" (UID: "6e852c03-a5bd-44b5-8792-ef2096c89a3f"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:14:44.968679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.968647    6389 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e852c03-a5bd-44b5-8792-ef2096c89a3f-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:44.968679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:44.968672    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lzr\" (UniqueName: \"kubernetes.io/projected/6e852c03-a5bd-44b5-8792-ef2096c89a3f-kube-api-access-h2lzr\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:45.415780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.415732    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:45.548654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.548607    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:14:45.629130 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.629083    6389 generic.go:356] "Generic (PLEG): container finished" podID="6e852c03-a5bd-44b5-8792-ef2096c89a3f" containerID="df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405" exitCode=0
Oct 30 20:14:45.629295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.629148    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"
Oct 30 20:14:45.629295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.629172    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc" event={"ID":"6e852c03-a5bd-44b5-8792-ef2096c89a3f","Type":"ContainerDied","Data":"df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405"}
Oct 30 20:14:45.629295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.629221    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc" event={"ID":"6e852c03-a5bd-44b5-8792-ef2096c89a3f","Type":"ContainerDied","Data":"24d8a9e4de2e01fa09993568356be115e9bc081bb2237d679b6144240351cc35"}
Oct 30 20:14:45.629295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.629245    6389 scope.go:122] "RemoveContainer" containerID="df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405"
Oct 30 20:14:45.638354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.638338    6389 scope.go:122] "RemoveContainer" containerID="df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405"
Oct 30 20:14:45.638595 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:14:45.638571    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405\": container with ID starting with df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405 not found: ID does not exist" containerID="df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405"
Oct 30 20:14:45.638637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.638605    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405"} err="failed to get container status \"df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405\": rpc error: code = NotFound desc = could not find container \"df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405\": container with ID starting with df80fdb2c71850fbb8ef53e78d125dd7f41258b4b7621bb9dbbc71ea9f001405 not found: ID does not exist"
Oct 30 20:14:45.651569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.651547    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"]
Oct 30 20:14:45.656969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.656950    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-2wfgc"]
Oct 30 20:14:45.912405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:14:45.912377    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e852c03-a5bd-44b5-8792-ef2096c89a3f" path="/var/lib/kubelet/pods/6e852c03-a5bd-44b5-8792-ef2096c89a3f/volumes"
Oct 30 20:14:55.790886 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:14:55.790850    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:15:01.876021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:01.875980    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:03.363105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:03.363065    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:04.817787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:04.817738    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:06.591970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:06.591909    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:08.470540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:08.470500    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:09.538917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:09.538878    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:10.564478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:10.564437    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:25.767067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:25.767022    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c8cf5b685-hv8q4"]
Oct 30 20:15:25.767657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:25.767235    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4" podUID="ca401ae1-d4da-4055-b3f9-46a532696d80" containerName="authorino" containerID="cri-o://3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994" gracePeriod=30
Oct 30 20:15:25.799666 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:15:25.799621    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:15:26.006482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.006459    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:15:26.023927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.023868    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert\" (UniqueName: \"kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert\") pod \"ca401ae1-d4da-4055-b3f9-46a532696d80\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") "
Oct 30 20:15:26.035547 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.035517    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert" pod "ca401ae1-d4da-4055-b3f9-46a532696d80" (UID: "ca401ae1-d4da-4055-b3f9-46a532696d80"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:15:26.124875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.124843    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq\" (UniqueName: \"kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq\") pod \"ca401ae1-d4da-4055-b3f9-46a532696d80\" (UID: \"ca401ae1-d4da-4055-b3f9-46a532696d80\") "
Oct 30 20:15:26.125166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.125146    6389 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca401ae1-d4da-4055-b3f9-46a532696d80-tls-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:15:26.126921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.126892    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq" pod "ca401ae1-d4da-4055-b3f9-46a532696d80" (UID: "ca401ae1-d4da-4055-b3f9-46a532696d80"). InnerVolumeSpecName "kube-api-access-r89lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:15:26.225545 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.225515    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r89lq\" (UniqueName: \"kubernetes.io/projected/ca401ae1-d4da-4055-b3f9-46a532696d80-kube-api-access-r89lq\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:15:26.343314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.343241    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:26.464715 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.464689    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:26.838045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.838013    6389 generic.go:356] "Generic (PLEG): container finished" podID="ca401ae1-d4da-4055-b3f9-46a532696d80" containerID="3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994" exitCode=0
Oct 30 20:15:26.838462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.838055    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4" event={"ID":"ca401ae1-d4da-4055-b3f9-46a532696d80","Type":"ContainerDied","Data":"3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994"}
Oct 30 20:15:26.838462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.838092    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4" event={"ID":"ca401ae1-d4da-4055-b3f9-46a532696d80","Type":"ContainerDied","Data":"339110cad589313f34cfb9b111642a8c0a2b2336ffcfa1431e7ec7ae8f0fcc97"}
Oct 30 20:15:26.838462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.838109    6389 scope.go:122] "RemoveContainer" containerID="3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994"
Oct 30 20:15:26.838462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.838061    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8cf5b685-hv8q4"
Oct 30 20:15:26.847335 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.847318    6389 scope.go:122] "RemoveContainer" containerID="3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994"
Oct 30 20:15:26.847624 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:15:26.847604    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994\": container with ID starting with 3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994 not found: ID does not exist" containerID="3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994"
Oct 30 20:15:26.847685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.847632    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994"} err="failed to get container status \"3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994\": rpc error: code = NotFound desc = could not find container \"3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994\": container with ID starting with 3b6c6565eba2584484de899749d1c556987227a58c0d19d901f67e1bef972994 not found: ID does not exist"
Oct 30 20:15:26.861625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.861605    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c8cf5b685-hv8q4"]
Oct 30 20:15:26.870673 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:26.870651    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c8cf5b685-hv8q4"]
Oct 30 20:15:27.913396 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:27.913360    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca401ae1-d4da-4055-b3f9-46a532696d80" path="/var/lib/kubelet/pods/ca401ae1-d4da-4055-b3f9-46a532696d80/volumes"
Oct 30 20:15:31.297440 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:31.297405    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:31.410577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:31.410523    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:33.256501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:33.256466    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:47.120668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.120638    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"]
Oct 30 20:15:47.121107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.120991    6389 cpu_manager.go:414] "containerMap: removing container" podUID="6e852c03-a5bd-44b5-8792-ef2096c89a3f" containerName="manager"
Oct 30 20:15:47.121107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.121003    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e852c03-a5bd-44b5-8792-ef2096c89a3f" containerName="manager"
Oct 30 20:15:47.121107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.121015    6389 cpu_manager.go:414] "containerMap: removing container" podUID="ca401ae1-d4da-4055-b3f9-46a532696d80" containerName="authorino"
Oct 30 20:15:47.121107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.121019    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca401ae1-d4da-4055-b3f9-46a532696d80" containerName="authorino"
Oct 30 20:15:47.121107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.121094    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e852c03-a5bd-44b5-8792-ef2096c89a3f" containerName="manager"
Oct 30 20:15:47.121107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.121102    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca401ae1-d4da-4055-b3f9-46a532696d80" containerName="authorino"
Oct 30 20:15:47.125723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.125705    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.139019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.138987    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"]
Oct 30 20:15:47.172260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172228    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172285    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172350    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172438    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172480    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172495    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172511    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172528    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hbvm\" (UniqueName: \"kubernetes.io/projected/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-kube-api-access-7hbvm\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.172739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.172653    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.223694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.223665    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9k4qv\""
Oct 30 20:15:47.226208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.226191    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 20:15:47.226208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.226199    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 20:15:47.226343 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.226238    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 20:15:47.273988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.273953    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.273988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.273990    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274028    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274053    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274069    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274084    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274104    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hbvm\" (UniqueName: \"kubernetes.io/projected/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-kube-api-access-7hbvm\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274154    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274191    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274642 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274609    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274656    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274720    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274811    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.274875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.274818    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.276458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.276432    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.276767 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.276751    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.287399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.287376    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.287572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.287518    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hbvm\" (UniqueName: \"kubernetes.io/projected/3323ebee-b8a0-47a1-94d2-d04a8bed6ffc-kube-api-access-7hbvm\") pod \"maas-default-gateway-openshift-default-64d4cc998c-ndkhp\" (UID: \"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.440813 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.440725    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:47.719101 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.719076    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"]
Oct 30 20:15:47.721110 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:15:47.721084    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3323ebee_b8a0_47a1_94d2_d04a8bed6ffc.slice/crio-4e096c7215e9b5d2c07b6113cb8566e5e1167f87da483eaced8e2f3dbbed4114 WatchSource:0}: Error finding container 4e096c7215e9b5d2c07b6113cb8566e5e1167f87da483eaced8e2f3dbbed4114: Status 404 returned error can't find the container with id 4e096c7215e9b5d2c07b6113cb8566e5e1167f87da483eaced8e2f3dbbed4114
Oct 30 20:15:47.938564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:47.938525    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" event={"ID":"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc","Type":"ContainerStarted","Data":"4e096c7215e9b5d2c07b6113cb8566e5e1167f87da483eaced8e2f3dbbed4114"}
Oct 30 20:15:50.618030 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:50.617998    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:15:50.618367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:50.618066    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:15:50.618367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:50.618095    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:15:50.957069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:50.956981    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" event={"ID":"3323ebee-b8a0-47a1-94d2-d04a8bed6ffc","Type":"ContainerStarted","Data":"5e7e0afe24939d681c2efe4d208ce122b9668713dacb61180edf863bbb98e290"}
Oct 30 20:15:50.983189 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:50.983107    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podStartSLOduration=1.088974676 podStartE2EDuration="3.983093655s" totalImagesPullingTime="2.894118979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:15:50.983023326 +0000 UTC m=+671.182596256" watchObservedRunningTime="2026-10-30 20:15:50.983093655 +0000 UTC m=+671.182666583"
Oct 30 20:15:51.444218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:51.444180    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:51.445560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:51.445216    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:51.445560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:51.445288    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:52.442952 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:52.442918    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:52.443618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:52.443576    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:53.445535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:53.445494    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:53.446161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:53.446095    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:54.441924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:54.441483    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:54.441924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:54.441542    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:55.446143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:55.441502    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:55.446143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:55.441564    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:55.791656 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:15:55.791621    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:15:56.051350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:56.051261    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:56.441951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:56.441861    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:56.441951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:56.441918    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:56.540417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:56.540382    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:57.442017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:57.441984    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 20:15:57.442225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:57.442039    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 20:15:58.198836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:58.198805    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:58.325815 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:58.325785    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:58.795100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:58.795071    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:58.795558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:58.795545    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:58.796188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:58.796171    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp"
Oct 30 20:15:59.420987 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:59.420955    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:59.520219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:59.520157    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:59.784974 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:59.784944    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:15:59.882276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:15:59.882244    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:02.176066 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.176035    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"]
Oct 30 20:16:02.180079 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.180055    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.199037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.199001    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"]
Oct 30 20:16:02.230787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.230753    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.230951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.230800    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.230951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.230829    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.231075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.230988    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.231075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.231050    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.231199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.231174    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-729hc\" (UniqueName: \"kubernetes.io/projected/8879a036-af1c-4d79-9897-633dd3a02edf-kube-api-access-729hc\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.231249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.231220    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879a036-af1c-4d79-9897-633dd3a02edf-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.231349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.231329    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879a036-af1c-4d79-9897-633dd3a02edf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.231403 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.231384    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879a036-af1c-4d79-9897-633dd3a02edf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332222 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332183    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879a036-af1c-4d79-9897-633dd3a02edf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332229    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879a036-af1c-4d79-9897-633dd3a02edf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332266    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332293    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332315    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332372    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332417    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332446    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-729hc\" (UniqueName: \"kubernetes.io/projected/8879a036-af1c-4d79-9897-633dd3a02edf-kube-api-access-729hc\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332475    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879a036-af1c-4d79-9897-633dd3a02edf-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.332924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.332880    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.333102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.333076    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.333195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.333113    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879a036-af1c-4d79-9897-633dd3a02edf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.333195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.333089    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.333345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.333323    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.334793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.334768    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879a036-af1c-4d79-9897-633dd3a02edf-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.335068 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.335052    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879a036-af1c-4d79-9897-633dd3a02edf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.347985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.347961    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879a036-af1c-4d79-9897-633dd3a02edf-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.348085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.348032    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-729hc\" (UniqueName: \"kubernetes.io/projected/8879a036-af1c-4d79-9897-633dd3a02edf-kube-api-access-729hc\") pod \"maas-default-gateway-openshift-default-64d4cc998c-g2nbq\" (UID: \"8879a036-af1c-4d79-9897-633dd3a02edf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.493638 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.493595    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:02.776271 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.776207    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"]
Oct 30 20:16:02.778293 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:02.778265    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8879a036_af1c_4d79_9897_633dd3a02edf.slice/crio-97511c662f208e20ca1c441246ac02275fbe2d77583e102ec5cd5e130d3098e2 WatchSource:0}: Error finding container 97511c662f208e20ca1c441246ac02275fbe2d77583e102ec5cd5e130d3098e2: Status 404 returned error can't find the container with id 97511c662f208e20ca1c441246ac02275fbe2d77583e102ec5cd5e130d3098e2
Oct 30 20:16:02.781443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.781409    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:16:02.781558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.781476    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:16:02.781558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:02.781506    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:16:03.015156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:03.015098    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" event={"ID":"8879a036-af1c-4d79-9897-633dd3a02edf","Type":"ContainerStarted","Data":"090b4a94495bf609964073dc5a5b83c55bbb4a3acd2abd769d9be0655ad9c8b3"}
Oct 30 20:16:03.015328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:03.015167    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" event={"ID":"8879a036-af1c-4d79-9897-633dd3a02edf","Type":"ContainerStarted","Data":"97511c662f208e20ca1c441246ac02275fbe2d77583e102ec5cd5e130d3098e2"}
Oct 30 20:16:03.042378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:03.042281    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podStartSLOduration=1.042266814 podStartE2EDuration="1.042266814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:03.040967335 +0000 UTC m=+683.240540267" watchObservedRunningTime="2026-10-30 20:16:03.042266814 +0000 UTC m=+683.241839756"
Oct 30 20:16:03.494089 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:03.494058    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:03.495719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:03.495691    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:03.495867 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:03.495746    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:04.494004 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:04.493970    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:04.494191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:04.494022    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:05.494769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:05.494734    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:05.495155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:05.494793    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:06.494927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:06.494892    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:06.495340 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:06.494958    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:07.494769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:07.494731    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:07.494948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:07.494795    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:08.494977 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:08.494939    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:08.495428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:08.495006    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:09.494016 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:09.493974    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 20:16:09.494214 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:09.494041    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 20:16:10.986412 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:10.986378    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:10.986959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:10.986776    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:10.987567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:10.987550    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq"
Oct 30 20:16:17.180293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.180254    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"]
Oct 30 20:16:17.184152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.184108    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.197198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.197173    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"]
Oct 30 20:16:17.268830 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.268797    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.268997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.268899    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.268997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.268916    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.268997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.268980    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.269100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.269031    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.269100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.269048    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.269100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.269084    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzhwj\" (UniqueName: \"kubernetes.io/projected/cf9639cd-63e1-4075-ac90-e4c5f7b25319-kube-api-access-qzhwj\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.269210 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.269192    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.269244 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.269216    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370514    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzhwj\" (UniqueName: \"kubernetes.io/projected/cf9639cd-63e1-4075-ac90-e4c5f7b25319-kube-api-access-qzhwj\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370567    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370596    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370627    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370699    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370726    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.370772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370746    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370796    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.370822    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.371336    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-data\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.371414    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-workload-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.371468    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-workload-certs\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.371501    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-credential-socket\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.371755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.371733    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.373206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.373180    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-envoy\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.373289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.373235    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-podinfo\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.384295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.384271    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9639cd-63e1-4075-ac90-e4c5f7b25319-istio-token\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.384500 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.384484    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzhwj\" (UniqueName: \"kubernetes.io/projected/cf9639cd-63e1-4075-ac90-e4c5f7b25319-kube-api-access-qzhwj\") pod \"maas-default-gateway-openshift-default-64d4cc998c-2kmq5\" (UID: \"cf9639cd-63e1-4075-ac90-e4c5f7b25319\") " pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.498197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.498164    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:17.778188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.778160    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"]
Oct 30 20:16:17.780604 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:17.780571    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9639cd_63e1_4075_ac90_e4c5f7b25319.slice/crio-f73cc704203c2cd7880dfc1c132da0f298de0b27ae17d9c97658743a5c8984aa WatchSource:0}: Error finding container f73cc704203c2cd7880dfc1c132da0f298de0b27ae17d9c97658743a5c8984aa: Status 404 returned error can't find the container with id f73cc704203c2cd7880dfc1c132da0f298de0b27ae17d9c97658743a5c8984aa
Oct 30 20:16:17.782655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.782620    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:16:17.782737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.782685    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:16:17.782737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:17.782713    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:16:18.112582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:18.112492    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" event={"ID":"cf9639cd-63e1-4075-ac90-e4c5f7b25319","Type":"ContainerStarted","Data":"ffd6049917c5b8ffcea4d75a341cbd61eebb501efeb369a4034ba43f14445fee"}
Oct 30 20:16:18.112582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:18.112562    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" event={"ID":"cf9639cd-63e1-4075-ac90-e4c5f7b25319","Type":"ContainerStarted","Data":"f73cc704203c2cd7880dfc1c132da0f298de0b27ae17d9c97658743a5c8984aa"}
Oct 30 20:16:18.138811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:18.138759    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podStartSLOduration=1.13874521 podStartE2EDuration="1.13874521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:18.13703858 +0000 UTC m=+698.336611510" watchObservedRunningTime="2026-10-30 20:16:18.13874521 +0000 UTC m=+698.338318138"
Oct 30 20:16:18.498805 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:18.498770    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:18.500229 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:18.500203    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:18.500351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:18.500263    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:19.498781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:19.498744    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:19.498960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:19.498806    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:20.499559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:20.499521    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:20.500061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:20.499596    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:21.498929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:21.498894    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:21.499112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:21.498944    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:22.498888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:22.498848    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:22.499356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:22.498904    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:23.463506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:23.463466    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:23.499482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:23.499454    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:23.499827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:23.499508    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:23.571754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:23.571720    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:23.673437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:23.673402    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.063072 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.063038    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.196341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.196298    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.381831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.381740    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.498838 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.498798    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:16:24.499002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.498863    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 20:16:24.500931 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.500905    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.624067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.624033    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.751366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.751321    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:24.880148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:24.880031    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:25.011819 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:25.011443    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:25.150795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:25.150763    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:25.792295 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:16:25.792259    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:16:26.278106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:26.278072    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:26.278637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:26.278609    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:26.280234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:26.280199    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5"
Oct 30 20:16:27.952033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:27.952002    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.064000 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.063959    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.238384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.238354    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.376665 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.376627    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.498916 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.498836    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.649144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.649079    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.820771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.820691    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:28.940242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:28.940203    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:29.558478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:29.558445    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:29.664959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:29.664931    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:30.366566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.366533    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:30.458186 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.458152    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"]
Oct 30 20:16:30.461549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.461530    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.470353 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.470317    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-44135b\""
Oct 30 20:16:30.470532 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.470508    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-44135b-dockercfg-mdst7\""
Oct 30 20:16:30.471878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.471855    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"]
Oct 30 20:16:30.485522 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.485497    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:30.596212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.596177    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.596628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.596220    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmkbm\" (UniqueName: \"kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm\") pod \"payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.598826 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.598800    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:30.697272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.697188    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.697272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.697229    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmkbm\" (UniqueName: \"kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm\") pod \"payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.697911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.697890    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.715789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.715755    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmkbm\" (UniqueName: \"kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm\") pod \"payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:30.775991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:30.775956    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:31.066314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.066289    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"]
Oct 30 20:16:31.068049 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:31.068007    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6747c8e_cf63_4849_aeee_dbbbcc43ef09.slice/crio-80d57e59c6ffeda52c04f3095fbb413aac389540dc5906390121204bad89d0c4 WatchSource:0}: Error finding container 80d57e59c6ffeda52c04f3095fbb413aac389540dc5906390121204bad89d0c4: Status 404 returned error can't find the container with id 80d57e59c6ffeda52c04f3095fbb413aac389540dc5906390121204bad89d0c4
Oct 30 20:16:31.171512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.171479    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:31.202365 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.202334    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" event={"ID":"d6747c8e-cf63-4849-aeee-dbbbcc43ef09","Type":"ContainerStarted","Data":"80d57e59c6ffeda52c04f3095fbb413aac389540dc5906390121204bad89d0c4"}
Oct 30 20:16:31.248889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.248853    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"]
Oct 30 20:16:31.253517 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.253499    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.262184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.262159    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-124fd1\""
Oct 30 20:16:31.262422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.262308    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-124fd1-dockercfg-8wl2q\""
Oct 30 20:16:31.262873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.262846    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"]
Oct 30 20:16:31.278291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.278269    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:31.281513 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.281482    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"]
Oct 30 20:16:31.284791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.284774    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.296545 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.296406    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"]
Oct 30 20:16:31.389170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.389065    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:31.403052 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.403023    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.403200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.403074    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xhz9\" (UniqueName: \"kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9\") pod \"payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.403245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.403204    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d47tr\" (UniqueName: \"kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr\") pod \"payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.403245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.403232    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.490039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.489971    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:31.504313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.504280    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.504458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.504335    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xhz9\" (UniqueName: \"kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9\") pod \"payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.504458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.504371    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d47tr\" (UniqueName: \"kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr\") pod \"payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.504458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.504397    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.505036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.505017    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.505159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.505110    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.519275 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.519244    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xhz9\" (UniqueName: \"kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9\") pod \"payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.519630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.519609    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d47tr\" (UniqueName: \"kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr\") pod \"payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.569269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.569239    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:31.599205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.599171    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:31.732701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.732670    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:31.775780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.775748    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"]
Oct 30 20:16:31.779903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.779876    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:31.786927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.786902    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-5197ec\""
Oct 30 20:16:31.789609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.789583    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"]
Oct 30 20:16:31.880346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.880308    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:31.891811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.891789    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"]
Oct 30 20:16:31.894288 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:31.894260    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb940e8_8ce9_4371_a732_0dcca00a0047.slice/crio-52755b1ac530dd99f86e45bc9d31776e0c4e17741f96e17816c5c62be3eb8c4a WatchSource:0}: Error finding container 52755b1ac530dd99f86e45bc9d31776e0c4e17741f96e17816c5c62be3eb8c4a: Status 404 returned error can't find the container with id 52755b1ac530dd99f86e45bc9d31776e0c4e17741f96e17816c5c62be3eb8c4a
Oct 30 20:16:31.904881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.904860    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"]
Oct 30 20:16:31.907167 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:31.907109    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40e1118_0922_4fbc_a1db_02244272ce81.slice/crio-2889741660bd078e03e034cc163960480d3675857db2ce51e39be0231a640b66 WatchSource:0}: Error finding container 2889741660bd078e03e034cc163960480d3675857db2ce51e39be0231a640b66: Status 404 returned error can't find the container with id 2889741660bd078e03e034cc163960480d3675857db2ce51e39be0231a640b66
Oct 30 20:16:31.907621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.907570    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzm6z\" (UniqueName: \"kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z\") pod \"payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:31.907667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:31.907614    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:32.008555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.008506    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzm6z\" (UniqueName: \"kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z\") pod \"payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:32.009234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.008837    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:32.009740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.009700    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:32.027063 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.027027    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzm6z\" (UniqueName: \"kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z\") pod \"payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:32.097234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.096990    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:32.214188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.214148    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" event={"ID":"b40e1118-0922-4fbc-a1db-02244272ce81","Type":"ContainerStarted","Data":"2889741660bd078e03e034cc163960480d3675857db2ce51e39be0231a640b66"}
Oct 30 20:16:32.215930 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.215898    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" event={"ID":"4eb940e8-8ce9-4371-a732-0dcca00a0047","Type":"ContainerStarted","Data":"52755b1ac530dd99f86e45bc9d31776e0c4e17741f96e17816c5c62be3eb8c4a"}
Oct 30 20:16:32.273398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.273284    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:32.344868 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.344831    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"]
Oct 30 20:16:32.349054 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.349029    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.357771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.357263    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-75e734-dockercfg-dq56s\""
Oct 30 20:16:32.360356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.360335    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"]
Oct 30 20:16:32.391498 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.391475    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:32.391997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.391976    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"]
Oct 30 20:16:32.394436 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:32.394405    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d15912_69fe_4e8f_8e0c_2634a8f6e4cb.slice/crio-352ec395914fc3cad0a526347583514c68cd297a0506386ce45112f1ba549edf WatchSource:0}: Error finding container 352ec395914fc3cad0a526347583514c68cd297a0506386ce45112f1ba549edf: Status 404 returned error can't find the container with id 352ec395914fc3cad0a526347583514c68cd297a0506386ce45112f1ba549edf
Oct 30 20:16:32.412714 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.412481    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.412714 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.412559    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhx8m\" (UniqueName: \"kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m\") pod \"payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.424689 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.424665    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-75e734\""
Oct 30 20:16:32.513652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.513613    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.513831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.513702    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhx8m\" (UniqueName: \"kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m\") pod \"payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.515021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.514996    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.536530 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.536500    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhx8m\" (UniqueName: \"kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m\") pod \"payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.583721 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.583686    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:32.668917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.668833    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:32.686472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.686402    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:32.789258 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.789226    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:32.970668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:32.970101    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"]
Oct 30 20:16:32.971936 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:32.971904    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ad98e6_2926_4c0b_8f77_2482f90e912b.slice/crio-a8dc28c57e3061d5e6c388d9f254fc3d521e5211c8dced8a5bcd17be56f6f9fd WatchSource:0}: Error finding container a8dc28c57e3061d5e6c388d9f254fc3d521e5211c8dced8a5bcd17be56f6f9fd: Status 404 returned error can't find the container with id a8dc28c57e3061d5e6c388d9f254fc3d521e5211c8dced8a5bcd17be56f6f9fd
Oct 30 20:16:33.122475 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.122402    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:33.175778 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.175745    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"]
Oct 30 20:16:33.181756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.181733    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.191720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.191683    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"]
Oct 30 20:16:33.192307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.192286    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-c646da\""
Oct 30 20:16:33.192436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.192420    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-c646da-dockercfg-4w4qh\""
Oct 30 20:16:33.232046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.231962    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" event={"ID":"c5ad98e6-2926-4c0b-8f77-2482f90e912b","Type":"ContainerStarted","Data":"a8dc28c57e3061d5e6c388d9f254fc3d521e5211c8dced8a5bcd17be56f6f9fd"}
Oct 30 20:16:33.234991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.234898    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" event={"ID":"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb","Type":"ContainerStarted","Data":"352ec395914fc3cad0a526347583514c68cd297a0506386ce45112f1ba549edf"}
Oct 30 20:16:33.241903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.241848    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:33.326329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.326291    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.326497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.326361    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjzp4\" (UniqueName: \"kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4\") pod \"payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.352326 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.352251    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:33.428144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.427046    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.428144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.427089    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjzp4\" (UniqueName: \"kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4\") pod \"payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.428144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.428075    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.452059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.452017    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjzp4\" (UniqueName: \"kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4\") pod \"payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.514870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.514586    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:33.790989 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.790860    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:33.874134 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.870276    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"]
Oct 30 20:16:33.879280 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:33.879220    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d494e7_3de8_4426_97f0_1cb29375fdce.slice/crio-b0c01665f0d04d4ba22262419f256acdd78f49510259947f0f04f3fad3fb00b8 WatchSource:0}: Error finding container b0c01665f0d04d4ba22262419f256acdd78f49510259947f0f04f3fad3fb00b8: Status 404 returned error can't find the container with id b0c01665f0d04d4ba22262419f256acdd78f49510259947f0f04f3fad3fb00b8
Oct 30 20:16:33.905416 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.904972    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:33.997156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:33.995465    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"]
Oct 30 20:16:34.001085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.001064    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.008204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.008137    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-909519-dockercfg-7bzmg\""
Oct 30 20:16:34.008332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.008234    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-909519\""
Oct 30 20:16:34.010139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.010003    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"]
Oct 30 20:16:34.020408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.020282    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:34.146096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.145815    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.146096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.145881    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6\") pod \"payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.246519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.246476    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6\") pod \"payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.246709 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.246651    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.247526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.247501    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.247641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.247553    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" event={"ID":"19d494e7-3de8-4426-97f0-1cb29375fdce","Type":"ContainerStarted","Data":"b0c01665f0d04d4ba22262419f256acdd78f49510259947f0f04f3fad3fb00b8"}
Oct 30 20:16:34.273366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.273313    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6\") pod \"payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.325065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.324787    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:34.656859 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.656232    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"]
Oct 30 20:16:34.753297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.748058    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"]
Oct 30 20:16:34.753297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.752597    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.760531 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.759740    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g52k9\""
Oct 30 20:16:34.763408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.762229    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 20:16:34.772589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.772550    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-c646da-serving-cert\""
Oct 30 20:16:34.775112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.774913    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"]
Oct 30 20:16:34.817498 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.817435    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:34.853489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.853447    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.853588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.853521    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.853588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.853551    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnlfj\" (UniqueName: \"kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.853671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.853622    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.916993 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.916907    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:34.954903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.954839    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.955061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.954909    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.955061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.954949    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.955061 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:16:34.955028    6389 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-serving-cert: secret "maas-api-e2e-worker-w0-c646da-serving-cert" not found
Oct 30 20:16:34.955337 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:16:34.955076    6389 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls podName:3c749eb2-04b4-4110-8367-c6e29548ec55 nodeName:}" failed. No retries permitted until 2026-10-30 20:16:35.455062245 +0000 UTC m=+715.654635151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls") pod "maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" (UID: "3c749eb2-04b4-4110-8367-c6e29548ec55") : secret "maas-api-e2e-worker-w0-c646da-serving-cert" not found
Oct 30 20:16:34.955851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.955573    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnlfj\" (UniqueName: \"kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.955851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.955727    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.958246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.958217    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:34.970941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:34.970921    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnlfj\" (UniqueName: \"kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:35.077776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.077745    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:35.255903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.255866    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" event={"ID":"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb","Type":"ContainerStarted","Data":"a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc"}
Oct 30 20:16:35.256040 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.255971    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:35.258246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.258213    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" event={"ID":"19d494e7-3de8-4426-97f0-1cb29375fdce","Type":"ContainerStarted","Data":"14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381"}
Oct 30 20:16:35.258360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.258343    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:35.261058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.260933    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" event={"ID":"d6747c8e-cf63-4849-aeee-dbbbcc43ef09","Type":"ContainerStarted","Data":"8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef"}
Oct 30 20:16:35.261058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.261020    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:35.263011 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.262982    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" event={"ID":"c5ad98e6-2926-4c0b-8f77-2482f90e912b","Type":"ContainerStarted","Data":"2d65b71b1fe1906001302a9322a06a1012fbe0778abf3f23b7d92caeaa3a2ef4"}
Oct 30 20:16:35.263575 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.263551    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:35.265164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.265141    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" event={"ID":"a9d4cbc9-d47a-45fb-b927-115458a1785b","Type":"ContainerStarted","Data":"f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb"}
Oct 30 20:16:35.265285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.265184    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" event={"ID":"a9d4cbc9-d47a-45fb-b927-115458a1785b","Type":"ContainerStarted","Data":"c2f5ed0e15c569488c539356d92a033f3261f68a52b89d9acf019d96424632ba"}
Oct 30 20:16:35.265285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.265229    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:35.276537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.276489    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" podStartSLOduration=1.870663635 podStartE2EDuration="4.276474058s" totalImagesPullingTime="2.405810423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:35.274412108 +0000 UTC m=+715.473985037" watchObservedRunningTime="2026-10-30 20:16:35.276474058 +0000 UTC m=+715.476046987"
Oct 30 20:16:35.295005 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.294965    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" podStartSLOduration=1.138508244 podStartE2EDuration="3.294953268s" totalImagesPullingTime="2.156445024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:35.294318317 +0000 UTC m=+715.493891246" watchObservedRunningTime="2026-10-30 20:16:35.294953268 +0000 UTC m=+715.494526232"
Oct 30 20:16:35.316514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.316460    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" podStartSLOduration=2.31644296 podStartE2EDuration="2.31644296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:35.314565721 +0000 UTC m=+715.514138653" watchObservedRunningTime="2026-10-30 20:16:35.31644296 +0000 UTC m=+715.516015888"
Oct 30 20:16:35.334471 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.334433    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" podStartSLOduration=1.82474585 podStartE2EDuration="5.33442026s" totalImagesPullingTime="3.50967441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:35.332879197 +0000 UTC m=+715.532452149" watchObservedRunningTime="2026-10-30 20:16:35.33442026 +0000 UTC m=+715.533993188"
Oct 30 20:16:35.356193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.356153    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" podStartSLOduration=1.463304553 podStartE2EDuration="2.356139305s" totalImagesPullingTime="892.834752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:35.355291 +0000 UTC m=+715.554863928" watchObservedRunningTime="2026-10-30 20:16:35.356139305 +0000 UTC m=+715.555712233"
Oct 30 20:16:35.461440 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.461416    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:35.463470 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.463449    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:35.678691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.678601    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:35.875503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.875472    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:35.959567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.959538    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"]
Oct 30 20:16:35.961935 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:35.961909    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c749eb2_04b4_4110_8367_c6e29548ec55.slice/crio-28ca5179c90b0573561f9b20d8c936f8d125a55eb49b17ac9784e8f9aa262fb4 WatchSource:0}: Error finding container 28ca5179c90b0573561f9b20d8c936f8d125a55eb49b17ac9784e8f9aa262fb4: Status 404 returned error can't find the container with id 28ca5179c90b0573561f9b20d8c936f8d125a55eb49b17ac9784e8f9aa262fb4
Oct 30 20:16:35.978199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:35.978147    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:36.091009 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.090966    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:36.273804 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.273767    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" event={"ID":"b40e1118-0922-4fbc-a1db-02244272ce81","Type":"ContainerStarted","Data":"c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466"}
Oct 30 20:16:36.274005 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.273940    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:36.275576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.275547    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" event={"ID":"4eb940e8-8ce9-4371-a732-0dcca00a0047","Type":"ContainerStarted","Data":"3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b"}
Oct 30 20:16:36.275716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.275605    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:36.276925 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.276900    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" event={"ID":"3c749eb2-04b4-4110-8367-c6e29548ec55","Type":"ContainerStarted","Data":"28ca5179c90b0573561f9b20d8c936f8d125a55eb49b17ac9784e8f9aa262fb4"}
Oct 30 20:16:36.296218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.296176    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" podStartSLOduration=1.8131193140000001 podStartE2EDuration="5.296163854s" totalImagesPullingTime="3.48304454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:36.293205084 +0000 UTC m=+716.492778012" watchObservedRunningTime="2026-10-30 20:16:36.296163854 +0000 UTC m=+716.495736783"
Oct 30 20:16:36.312030 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.311991    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" podStartSLOduration=1.85283029 podStartE2EDuration="5.311981062s" totalImagesPullingTime="3.459150772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:36.311272527 +0000 UTC m=+716.510845455" watchObservedRunningTime="2026-10-30 20:16:36.311981062 +0000 UTC m=+716.511553989"
Oct 30 20:16:36.948886 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:36.948851    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:37.053462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:37.053428    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:37.160132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:37.160098    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:38.124316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:38.124281    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:39.298349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:39.298311    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" event={"ID":"3c749eb2-04b4-4110-8367-c6e29548ec55","Type":"ContainerStarted","Data":"7827116243c22279e5d72a569c9b08479a8328181610bf43161d9adf91c789d2"}
Oct 30 20:16:39.298849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:39.298413    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:39.320652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:39.320603    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" podStartSLOduration=2.7441303870000002 podStartE2EDuration="5.320566567s" totalImagesPullingTime="2.57643618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:39.319556723 +0000 UTC m=+719.519129656" watchObservedRunningTime="2026-10-30 20:16:39.320566567 +0000 UTC m=+719.520139520"
Oct 30 20:16:39.358864 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:39.358824    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:40.619069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:40.619037    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:41.544199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:41.544163    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:42.890559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:42.890508    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:43.197226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:43.197104    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:43.299399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:43.299352    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:43.822421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:43.822387    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:43.933468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:43.933433    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:44.047313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:44.047279    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:44.161398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:44.161307    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:44.360294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:44.360248    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:44.987514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:44.987483    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"]
Oct 30 20:16:44.992586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:44.992562    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.002281 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.002256    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-44135b\"/\"kube-root-ca.crt\""
Oct 30 20:16:45.002430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.002309    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-44135b\"/\"openshift-service-ca.crt\""
Oct 30 20:16:45.002430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.002319    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-44135b\"/\"default-dockercfg-7wx9c\""
Oct 30 20:16:45.002430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.002256    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-44135b\"/\"simulator-w5-44135b-kserve-self-signed-certs\""
Oct 30 20:16:45.004143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.004093    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"]
Oct 30 20:16:45.047327 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.047296    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:45.151752 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.151722    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.151913 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.151864    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.151913 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.151887    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.151986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.151914    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.151986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.151937    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.152052 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.151995    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfqv7\" (UniqueName: \"kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.252667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.252594    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.252667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.252633    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.252889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.252669    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.252889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.252699    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.252889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.252722    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfqv7\" (UniqueName: \"kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.252889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.252748    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.253099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.253055    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.253099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.253063    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.253244 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.253163    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.255056 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.255034    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.255337 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.255319    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.265713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.265692    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfqv7\" (UniqueName: \"kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7\") pod \"simulator-w5-44135b-kserve-5fcb85c46f-8gc96\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") " pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.305461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.305434    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:45.309906 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.309425    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:16:45.539312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.539227    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:45.593736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:45.593685    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"]
Oct 30 20:16:45.596176 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:45.596147    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbefeed63_0326_4723_9bfa_0eb4535aaae1.slice/crio-bf7e37de848abbfb7fddf6bcdad86c0fbd235572da7465a4459665a1faf3f8e9 WatchSource:0}: Error finding container bf7e37de848abbfb7fddf6bcdad86c0fbd235572da7465a4459665a1faf3f8e9: Status 404 returned error can't find the container with id bf7e37de848abbfb7fddf6bcdad86c0fbd235572da7465a4459665a1faf3f8e9
Oct 30 20:16:46.281042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.281016    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:16:46.282175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.282156    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:16:46.282696 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.282678    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:16:46.283865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.283576    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:16:46.287591 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.287568    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"]
Oct 30 20:16:46.294345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.294324    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.301569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.301543    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"]
Oct 30 20:16:46.301750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.301731    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-124fd1\"/\"kube-root-ca.crt\""
Oct 30 20:16:46.306871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.306482    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-124fd1\"/\"openshift-service-ca.crt\""
Oct 30 20:16:46.310621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.307964    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-124fd1\"/\"default-dockercfg-v2g4p\""
Oct 30 20:16:46.310621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.310385    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-124fd1\"/\"simulator-w3-124fd1-kserve-self-signed-certs\""
Oct 30 20:16:46.350092 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.350047    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:46.356137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.356006    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" event={"ID":"befeed63-0326-4723-9bfa-0eb4535aaae1","Type":"ContainerStarted","Data":"bf7e37de848abbfb7fddf6bcdad86c0fbd235572da7465a4459665a1faf3f8e9"}
Oct 30 20:16:46.467504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.467467    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.467657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.467553    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.467657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.467594    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.467730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.467703    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.467800 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.467785    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.467835 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.467819    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgdkg\" (UniqueName: \"kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.568938    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.568983    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569104    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569205    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569246    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569273    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgdkg\" (UniqueName: \"kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569479    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569552 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569516    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.569766 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.569740    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.571509 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.571485    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.571957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.571934    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.582958 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.582928    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgdkg\" (UniqueName: \"kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg\") pod \"simulator-w3-124fd1-kserve-b58d7c57b-gztfp\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") " pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.616863 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.616694    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:46.906214 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.906183    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"]
Oct 30 20:16:46.910620 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:46.910588    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf1c2b2c_022e_4125_8d0f_112af9b9ee5b.slice/crio-55c54d8e69f971fd540ec9b2f701cb925a25b92fd8f42f2295f76d490ddd8045 WatchSource:0}: Error finding container 55c54d8e69f971fd540ec9b2f701cb925a25b92fd8f42f2295f76d490ddd8045: Status 404 returned error can't find the container with id 55c54d8e69f971fd540ec9b2f701cb925a25b92fd8f42f2295f76d490ddd8045
Oct 30 20:16:46.978630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:46.978596    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:47.286487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.286465    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:16:47.286816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.286539    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:16:47.286816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.286585    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:16:47.367155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.366270    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" event={"ID":"befeed63-0326-4723-9bfa-0eb4535aaae1","Type":"ContainerStarted","Data":"b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133"}
Oct 30 20:16:47.369193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.369158    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" event={"ID":"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b","Type":"ContainerStarted","Data":"24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4"}
Oct 30 20:16:47.369348 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.369205    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" event={"ID":"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b","Type":"ContainerStarted","Data":"55c54d8e69f971fd540ec9b2f701cb925a25b92fd8f42f2295f76d490ddd8045"}
Oct 30 20:16:47.388100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.388046    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podStartSLOduration=1.9282145320000001 podStartE2EDuration="3.388028573s" totalImagesPullingTime="1.459814041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:47.38731363 +0000 UTC m=+727.586886564" watchObservedRunningTime="2026-10-30 20:16:47.388028573 +0000 UTC m=+727.587601502"
Oct 30 20:16:47.420431 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.420382    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podStartSLOduration=1.105850564 podStartE2EDuration="1.42036592s" totalImagesPullingTime="314.515356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:47.41889557 +0000 UTC m=+727.618468498" watchObservedRunningTime="2026-10-30 20:16:47.42036592 +0000 UTC m=+727.619938849"
Oct 30 20:16:47.617589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.617505    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:47.799057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.797389    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"]
Oct 30 20:16:47.801762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.801734    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.809025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.808669    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5197ec\"/\"simulator-w1-5197ec-kserve-self-signed-certs\""
Oct 30 20:16:47.812731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.810337    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5197ec\"/\"default-dockercfg-2nnss\""
Oct 30 20:16:47.812731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.811184    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5197ec\"/\"kube-root-ca.crt\""
Oct 30 20:16:47.812731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.811292    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5197ec\"/\"openshift-service-ca.crt\""
Oct 30 20:16:47.812731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.811871    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"]
Oct 30 20:16:47.851436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.851405    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:47.889015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.888930    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.889015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.888994    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.889246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.889054    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.889246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.889084    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnkdr\" (UniqueName: \"kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.889246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.889163    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.889359 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.889275    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.990634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.990602    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.990822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.990644    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.990822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.990676    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnkdr\" (UniqueName: \"kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.990822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.990699    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.990822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.990740    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.990822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.990816    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.991200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.991175    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.991279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.991202    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.991340 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.991323    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.993500 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.993478    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:47.993863 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:47.993841    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:48.013889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.013864    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnkdr\" (UniqueName: \"kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr\") pod \"simulator-w1-5197ec-kserve-597d8f7854-m5r5j\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") " pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:48.116208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.116174    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:48.306311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.306280    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:48.400523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.400483    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"]
Oct 30 20:16:48.405254 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:48.405226    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dbbcef9_bccf_4cfe_abe8_0a0f9cea4a65.slice/crio-eca93c18264f87f469fc44ed1360ba66d1f83ba1b210b0c72961f75936a4009f WatchSource:0}: Error finding container eca93c18264f87f469fc44ed1360ba66d1f83ba1b210b0c72961f75936a4009f: Status 404 returned error can't find the container with id eca93c18264f87f469fc44ed1360ba66d1f83ba1b210b0c72961f75936a4009f
Oct 30 20:16:48.568397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.568367    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:48.673865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.673752    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:48.778142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:48.778079    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:49.192627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.192595    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:49.321062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.321029    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:49.383775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.383738    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" event={"ID":"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65","Type":"ContainerStarted","Data":"00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa"}
Oct 30 20:16:49.383775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.383781    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" event={"ID":"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65","Type":"ContainerStarted","Data":"eca93c18264f87f469fc44ed1360ba66d1f83ba1b210b0c72961f75936a4009f"}
Oct 30 20:16:49.405944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.405897    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podStartSLOduration=2.405882623 podStartE2EDuration="2.405882623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:49.403280381 +0000 UTC m=+729.602853310" watchObservedRunningTime="2026-10-30 20:16:49.405882623 +0000 UTC m=+729.605455550"
Oct 30 20:16:49.433459 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.433418    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:49.836503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.836459    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:49.947154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:49.947096    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:50.059699 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:50.059668    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:50.116825 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:50.116751    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:51.078384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:51.078308    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:52.197396 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:52.197364    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:53.130895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:53.130865    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:54.072448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:54.072412    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:55.116590 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:55.116559    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:55.306236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:55.306192    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:55.319251 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:55.319226    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:55.433662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:55.433588    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:16:55.804191 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:16:55.804100    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:16:55.833560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:55.833531    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:56.344561 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:56.344528    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:56.617648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:56.617574    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:56.630690 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:56.630662    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:57.441937 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:57.441909    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:16:57.467663 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:57.467626    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:58.117078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:58.117046    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:58.134465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:58.134439    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:58.390828 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:58.390746    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:58.449617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:58.449584    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:16:58.893520 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:58.893491    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:59.088834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.088802    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"]
Oct 30 20:16:59.092799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.092769    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.103070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.102694    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-909519\"/\"kube-root-ca.crt\""
Oct 30 20:16:59.103070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.102846    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-909519\"/\"simulator-w2-909519-kserve-self-signed-certs\""
Oct 30 20:16:59.103070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.102871    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-909519\"/\"default-dockercfg-p2vpb\""
Oct 30 20:16:59.103070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.102921    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-909519\"/\"openshift-service-ca.crt\""
Oct 30 20:16:59.105331 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.105309    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"]
Oct 30 20:16:59.147416 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.147331    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:59.205588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.205553    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.205778 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.205605    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.205778 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.205727    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.205908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.205830    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.205908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.205861    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbddt\" (UniqueName: \"kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.206038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.206017    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.306881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.306848    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.306914    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.306942    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.306959    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.306995    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.307019    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbddt\" (UniqueName: \"kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307515 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.307484    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.307550    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.307691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.307673    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.309447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.309417    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.309718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.309700    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.323403 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.323343    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbddt\" (UniqueName: \"kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt\") pod \"simulator-w2-909519-kserve-86fb7498b-mp65m\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") " pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.408414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.408332    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:16:59.418357 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.418312    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:59.537388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.537353    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:59.658661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.658570    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:16:59.708701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.708673    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"]
Oct 30 20:16:59.710392 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:16:59.710362    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d9bb5a6_2863_4b96_92d0_5b5579f20074.slice/crio-019c84573f40517b07f2326060db32898a69a707fa9a61ec3b4994ffd3cf85e1 WatchSource:0}: Error finding container 019c84573f40517b07f2326060db32898a69a707fa9a61ec3b4994ffd3cf85e1: Status 404 returned error can't find the container with id 019c84573f40517b07f2326060db32898a69a707fa9a61ec3b4994ffd3cf85e1
Oct 30 20:16:59.791413 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:16:59.791381    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:00.453310 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:00.453272    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" event={"ID":"3d9bb5a6-2863-4b96-92d0-5b5579f20074","Type":"ContainerStarted","Data":"07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea"}
Oct 30 20:17:00.453502 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:00.453324    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" event={"ID":"3d9bb5a6-2863-4b96-92d0-5b5579f20074","Type":"ContainerStarted","Data":"019c84573f40517b07f2326060db32898a69a707fa9a61ec3b4994ffd3cf85e1"}
Oct 30 20:17:00.474421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:00.474367    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podStartSLOduration=1.4743536910000001 podStartE2EDuration="1.474353691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:00.472725977 +0000 UTC m=+740.672298899" watchObservedRunningTime="2026-10-30 20:17:00.474353691 +0000 UTC m=+740.673926618"
Oct 30 20:17:00.596854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:00.596818    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:01.407544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:01.407513    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:01.408416 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:01.408397    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:17:02.103264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:02.103228    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:02.459218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:02.459141    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:03.362351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:03.362319    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:04.146905 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:04.146873    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:05.085908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:05.085878    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:05.186590 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:05.186556    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:06.010810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:06.010781    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:07.018238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.018202    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:07.721292 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.721239    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:07.884304 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.884268    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"]
Oct 30 20:17:07.891845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.891820    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:07.899093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.899058    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"]
Oct 30 20:17:07.899279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.899181    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-c646da\"/\"kube-root-ca.crt\""
Oct 30 20:17:07.899279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.899243    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-c646da\"/\"openshift-service-ca.crt\""
Oct 30 20:17:07.899845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.899820    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c646da\"/\"simulator-w0-c646da-kserve-self-signed-certs\""
Oct 30 20:17:07.899944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.899928    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c646da\"/\"default-dockercfg-77sdv\""
Oct 30 20:17:07.936320 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.936279    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:07.994623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.994536    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:07.994623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.994602    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:07.994808 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.994731    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:07.994808 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.994784    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:07.994876 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.994850    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:07.994924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:07.994910    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgdmn\" (UniqueName: \"kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.095860 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.095824    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.095885    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.095914    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.095935    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.095956    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.095989    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgdmn\" (UniqueName: \"kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.096470    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.096488    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.096726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.096527    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.098446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.098420    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.098703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.098681    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.109351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.109328    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgdmn\" (UniqueName: \"kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn\") pod \"simulator-w0-c646da-kserve-557544f9b7-rq55t\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") " pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.209718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.209681    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:08.487762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.487737    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"]
Oct 30 20:17:08.489629 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:17:08.489596    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1029303a_e800_4bf0_be8c_bbe05fcec90d.slice/crio-5403536f789c6dcd23ca89d82b93f244b819074f341d4d9ad254e2ff17f3292b WatchSource:0}: Error finding container 5403536f789c6dcd23ca89d82b93f244b819074f341d4d9ad254e2ff17f3292b: Status 404 returned error can't find the container with id 5403536f789c6dcd23ca89d82b93f244b819074f341d4d9ad254e2ff17f3292b
Oct 30 20:17:08.498895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.498864    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" event={"ID":"1029303a-e800-4bf0-be8c-bbe05fcec90d","Type":"ContainerStarted","Data":"5403536f789c6dcd23ca89d82b93f244b819074f341d4d9ad254e2ff17f3292b"}
Oct 30 20:17:08.562935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:08.562902    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:09.408634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:09.408598    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:17:09.421375 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:09.421342    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:17:09.476214 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:09.476162    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:09.506566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:09.506533    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" event={"ID":"1029303a-e800-4bf0-be8c-bbe05fcec90d","Type":"ContainerStarted","Data":"5ad75c7182259906f37d5650003db3e6a01e45fb333da2f031b8117dc9e5ec52"}
Oct 30 20:17:09.518706 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:09.518661    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:17:09.527389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:09.527328    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podStartSLOduration=2.527291451 podStartE2EDuration="2.527291451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:09.526729533 +0000 UTC m=+749.726302456" watchObservedRunningTime="2026-10-30 20:17:09.527291451 +0000 UTC m=+749.726864380"
Oct 30 20:17:10.209837 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:10.209805    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:10.364046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:10.364018    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:11.258487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:11.258455    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:12.080762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:12.080705    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:12.515153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:12.515092    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:12.957317 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:12.957220    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:13.688518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:13.688480    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:14.570472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:14.570438    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:15.291350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.291308    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:15.492505 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.492473    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"]
Oct 30 20:17:15.499711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.499683    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.505474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.505445    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-124fd1\"/\"premium-w3-124fd1-kserve-self-signed-certs\""
Oct 30 20:17:15.508382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.508354    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"]
Oct 30 20:17:15.541878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.541796    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:15.672757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.672724    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.672955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.672772    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.672955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.672807    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4twl8\" (UniqueName: \"kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.673065 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.673036    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.673136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.673088    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.673203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.673185    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774361 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774325    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774372    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774410    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774446    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774490    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774755    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774795    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774892 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774533    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4twl8\" (UniqueName: \"kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.774986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.774964    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.776944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.776911    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.777899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.777870    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.788159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.788110    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4twl8\" (UniqueName: \"kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8\") pod \"premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") " pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:15.817833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:15.817762    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:16.099193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:16.099171    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"]
Oct 30 20:17:16.101497 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:17:16.101469    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod831bc213_3f56_42f5_88a2_12d399c1a81e.slice/crio-97a0f1bb1d5d836b7cbe292e534211bbad88ad5c593afc41080afedfe29fa2f3 WatchSource:0}: Error finding container 97a0f1bb1d5d836b7cbe292e534211bbad88ad5c593afc41080afedfe29fa2f3: Status 404 returned error can't find the container with id 97a0f1bb1d5d836b7cbe292e534211bbad88ad5c593afc41080afedfe29fa2f3
Oct 30 20:17:16.262397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:16.262365    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:16.552751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:16.552715    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" event={"ID":"831bc213-3f56-42f5-88a2-12d399c1a81e","Type":"ContainerStarted","Data":"bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa"}
Oct 30 20:17:16.553223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:16.552761    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" event={"ID":"831bc213-3f56-42f5-88a2-12d399c1a81e","Type":"ContainerStarted","Data":"97a0f1bb1d5d836b7cbe292e534211bbad88ad5c593afc41080afedfe29fa2f3"}
Oct 30 20:17:16.579686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:16.579630    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podStartSLOduration=1.5796037 podStartE2EDuration="1.5796037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:16.576072383 +0000 UTC m=+756.775645313" watchObservedRunningTime="2026-10-30 20:17:16.5796037 +0000 UTC m=+756.779176630"
Oct 30 20:17:16.818961 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:16.818883    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:17.055896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:17.055858    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:17.940913 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:17.940877    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:18.210357 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:18.210261    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:18.229718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:18.229691    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:18.577585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:18.577559    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:17:18.631413 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:18.631370    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:18.769033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:18.768997    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:19.469753 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:19.469720    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:20.259434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:20.259401    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:21.176322 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:21.176277    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:21.978394 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:21.978366    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:22.777329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:22.777299    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:23.682865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:23.682818    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:24.399638 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:24.399583    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:25.306291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.306242    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:25.453438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.453408    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"]
Oct 30 20:17:25.457298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.457275    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.465627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.465430    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-75e734\"/\"kube-root-ca.crt\""
Oct 30 20:17:25.465627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.465465    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-75e734\"/\"default-dockercfg-kftsm\""
Oct 30 20:17:25.465627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.465588    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-75e734\"/\"openshift-service-ca.crt\""
Oct 30 20:17:25.466177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.466158    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-75e734\"/\"premium-w6-75e734-kserve-self-signed-certs\""
Oct 30 20:17:25.470233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.470208    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"]
Oct 30 20:17:25.511132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.511093    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:25.575854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.575761    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.575854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.575822    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.576083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.575919    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwnv\" (UniqueName: \"kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.576083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.575996    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.576083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.576031    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.576275 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.576190    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677069    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwnv\" (UniqueName: \"kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677106    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677147    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677258    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677332    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677522 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677376    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677596    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677622    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.677679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.677647    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.679843 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.679817    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.679969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.679947    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.696333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.690572    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwnv\" (UniqueName: \"kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv\") pod \"premium-w6-75e734-kserve-6665fdf8c7-fprn2\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") " pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.771679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.771647    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:25.793419 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:17:25.793386    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:17:25.818775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.818750    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:25.845383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:25.845358    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:26.062820 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.062789    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"]
Oct 30 20:17:26.063200 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:17:26.063178    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ab09e66_6512_431a_b3ab_0358b57baeac.slice/crio-53e599f1bfab8f47ff500d646ae99edd12328c5eefedc041cf6c1d065d7555df WatchSource:0}: Error finding container 53e599f1bfab8f47ff500d646ae99edd12328c5eefedc041cf6c1d065d7555df: Status 404 returned error can't find the container with id 53e599f1bfab8f47ff500d646ae99edd12328c5eefedc041cf6c1d065d7555df
Oct 30 20:17:26.083386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.083360    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:26.616650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.616611    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" event={"ID":"7ab09e66-6512-431a-b3ab-0358b57baeac","Type":"ContainerStarted","Data":"296c7e28e24a941ad5e2e6f39dc419c2b965e848bf195bf43351093486bacc2a"}
Oct 30 20:17:26.617108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.616659    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" event={"ID":"7ab09e66-6512-431a-b3ab-0358b57baeac","Type":"ContainerStarted","Data":"53e599f1bfab8f47ff500d646ae99edd12328c5eefedc041cf6c1d065d7555df"}
Oct 30 20:17:26.628081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.628057    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:17:26.640589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.640549    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podStartSLOduration=1.640537028 podStartE2EDuration="1.640537028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:26.637639309 +0000 UTC m=+766.837212239" watchObservedRunningTime="2026-10-30 20:17:26.640537028 +0000 UTC m=+766.840109955"
Oct 30 20:17:26.772153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.772102    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:26.924141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:26.924044    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:27.659544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:27.659516    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:28.075204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:28.075162    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:28.450346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:28.450267    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:29.470461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:29.470427    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:30.126916 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:30.126873    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:30.227008 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:30.226970    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:30.334102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:30.334068    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:31.043035 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:31.043005    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:31.749598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:31.749572    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:32.188843 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:32.188766    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:32.668644 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:32.668608    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:33.371387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:33.371356    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:33.668422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:33.668345    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:34.365352 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.365322    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:34.479199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.479169    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"]
Oct 30 20:17:34.483812 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.483789    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.491463 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.491434    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-9c901b\"/\"openshift-service-ca.crt\""
Oct 30 20:17:34.491591 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.491573    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-9c901b\"/\"kube-root-ca.crt\""
Oct 30 20:17:34.501179 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.492205    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-9c901b\"/\"default-dockercfg-m7fvr\""
Oct 30 20:17:34.501179 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.492301    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-9c901b\"/\"premium-w4-9c901b-kserve-self-signed-certs\""
Oct 30 20:17:34.501179 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.496578    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"]
Oct 30 20:17:34.529666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.529638    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:34.569441 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.569406    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.569619 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.569510    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89gs4\" (UniqueName: \"kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.569619 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.569588    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.569802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.569673    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.569802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.569781    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.569903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.569826    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.670720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.670632    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.670720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.670691    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.670955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.670741    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.670955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.670777    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.670955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.670802    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89gs4\" (UniqueName: \"kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.670955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.670824    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.671205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.671077    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.671205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.671148    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.671205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.671197    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.673479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.673450    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.673704 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.673684    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.684188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.684154    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89gs4\" (UniqueName: \"kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4\") pod \"premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") " pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:34.814175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:34.814138    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:35.092686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.092648    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:35.094192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.094170    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"]
Oct 30 20:17:35.095957 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:17:35.095927    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc447eceb_5c07_494f_872e_979635d9fd83.slice/crio-e27bba1a8a8fdf7425ca23db494d2777240728ae2cdb0891cca7b30a9862bd6f WatchSource:0}: Error finding container e27bba1a8a8fdf7425ca23db494d2777240728ae2cdb0891cca7b30a9862bd6f: Status 404 returned error can't find the container with id e27bba1a8a8fdf7425ca23db494d2777240728ae2cdb0891cca7b30a9862bd6f
Oct 30 20:17:35.674353 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.674294    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" event={"ID":"c447eceb-5c07-494f-872e-979635d9fd83","Type":"ContainerStarted","Data":"ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73"}
Oct 30 20:17:35.674844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.674367    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" event={"ID":"c447eceb-5c07-494f-872e-979635d9fd83","Type":"ContainerStarted","Data":"e27bba1a8a8fdf7425ca23db494d2777240728ae2cdb0891cca7b30a9862bd6f"}
Oct 30 20:17:35.696104 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.696059    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podStartSLOduration=1.69604519 podStartE2EDuration="1.69604519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:35.694206162 +0000 UTC m=+775.893779090" watchObservedRunningTime="2026-10-30 20:17:35.69604519 +0000 UTC m=+775.895618131"
Oct 30 20:17:35.772316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.772286    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:35.784773 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.784748    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:35.815024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.815004    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:35.962476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:35.962404    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:36.691577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:36.691548    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:17:36.692805 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:36.692779    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:37.382906 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:37.382873    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:38.514113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:38.514056    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:39.429265 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:39.429230    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:39.991494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:39.991468    6389 scope.go:122] "RemoveContainer" containerID="6d9f7e0cb61aadc497be37a7eaf5223022a23a589131fe8b1323724abfe89187"
Oct 30 20:17:40.223939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:40.223891    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:40.930152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:40.930105    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:41.781263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:41.781231    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:42.561362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:42.561319    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:43.344191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:43.344153    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:44.078315 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:44.078277    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:44.815181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:44.815149    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:44.824262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:44.824235    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:44.827733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:44.827716    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:45.745546 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:45.745517    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:17:45.956931 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:45.956898    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:47.065177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:47.065146    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:47.975702 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:47.975671    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:49.056972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:49.056938    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:49.868422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:49.868387    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:50.829075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:50.829041    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:51.556991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:51.556960    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:52.368045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:52.368012    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:53.071550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:53.071515    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:53.780357 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:53.780318    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:54.672156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.672104    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:54.845476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.845445    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"]
Oct 30 20:17:54.849034 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.849016    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.853625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.853604    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-44135b-openshift-default-dockercfg-sp9bd\""
Oct 30 20:17:54.860513 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.860488    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"]
Oct 30 20:17:54.873471 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873444    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.873581 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873495    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.873659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873636    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.873812 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873796    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.873882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873837    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.873935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873916    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.873976 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873945    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgftb\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.874020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873980    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.874020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.873997    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.974737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.974681    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.974910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.974743    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.974910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.974798    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.974917    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgftb\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.974972    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.974997    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.975057    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.975112    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.975160    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.975524 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.975503    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.976567 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.975560    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.976718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.976200    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.976718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.976248    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.976851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.976666    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.978005 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.977982    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.978284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.978267    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.989178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.989153    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:54.989596 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:54.989577    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgftb\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:55.163883 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.163854    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:55.445750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.445722    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"]
Oct 30 20:17:55.448407 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:17:55.448377    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2e82a7_b61b_452e_b956_a9173e4eef49.slice/crio-d8d5c642709562b1994496ae067b716346de261c1d769c10712040d3abe356e3 WatchSource:0}: Error finding container d8d5c642709562b1994496ae067b716346de261c1d769c10712040d3abe356e3: Status 404 returned error can't find the container with id d8d5c642709562b1994496ae067b716346de261c1d769c10712040d3abe356e3
Oct 30 20:17:55.451791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.451762    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:17:55.451864 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.451824    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:17:55.451864 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.451852    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:17:55.479299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.479244    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:55.797245 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:17:55.797205    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:17:55.802156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.802101    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" event={"ID":"0c2e82a7-b61b-452e-b956-a9173e4eef49","Type":"ContainerStarted","Data":"ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d"}
Oct 30 20:17:55.802324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.802219    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" event={"ID":"0c2e82a7-b61b-452e-b956-a9173e4eef49","Type":"ContainerStarted","Data":"d8d5c642709562b1994496ae067b716346de261c1d769c10712040d3abe356e3"}
Oct 30 20:17:55.828719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:55.828667    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podStartSLOduration=1.8286539290000001 podStartE2EDuration="1.828653929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:55.826722506 +0000 UTC m=+796.026295435" watchObservedRunningTime="2026-10-30 20:17:55.828653929 +0000 UTC m=+796.028226856"
Oct 30 20:17:56.165169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:56.165049    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:17:56.166380 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:56.166357    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 20:17:56.166466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:56.166416    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 20:17:56.246935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:56.246891    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:57.164980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:57.164944    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 20:17:57.165191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:57.165006    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 20:17:57.191022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:57.190992    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:58.165030 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:58.164991    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 20:17:58.165289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:58.165071    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 20:17:58.737671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:58.737610    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:17:59.164587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:59.164503    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 20:17:59.164587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:59.164569    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 20:17:59.700497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:17:59.700466    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:00.164468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:00.164432    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 20:18:00.164641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:00.164484    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 20:18:00.575646 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:00.575605    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:01.164924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:01.164894    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 20:18:01.165354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:01.164957    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 20:18:01.456708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:01.456625    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:02.515774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:02.515740    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:02.881911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:02.881828    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:18:02.882355 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:02.882338    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:18:02.883225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:02.883204    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:18:03.357892 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:03.357862    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:04.360392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:04.360356    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:05.159407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:05.159369    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:06.064589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:06.064561    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:06.855401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:06.855369    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:07.865454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:07.865421    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:08.580271 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:08.580238    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:09.296283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.296254    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:09.542844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.542810    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"]
Oct 30 20:18:09.548289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.548225    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.560508 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.560481    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"]
Oct 30 20:18:09.640212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.640181    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-c646da-openshift-default-dockercfg-2gm5m\""
Oct 30 20:18:09.656155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656101    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656160    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbtz\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656294    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656341    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656392    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656415    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656509 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656489    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656520    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.656587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.656580    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.668028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.668003    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"]
Oct 30 20:18:09.673156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.673135    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.694961 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.694928    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"]
Oct 30 20:18:09.740296 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.740265    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-9c901b-openshift-default-dockercfg-hnkkz\""
Oct 30 20:18:09.757799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.757769    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.757946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.757821    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.757946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.757844    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758000    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758045    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758072    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbtz\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758111    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758231 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758214    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758287 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758246    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758306    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758346    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758403    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758433    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khszw\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758496    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758529    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758558    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758573    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758596    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.758628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758624    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758697    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.758858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.758841    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.759074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.759043    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.759218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.759174    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.760928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.760899    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.761338 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.761319    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.772391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.772367    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbtz\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.772945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.772920    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.860260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860173    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860210    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860230    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khszw\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860271    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860300    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860344    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860383    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860418    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860481    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860866 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860795    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.860866 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.860835    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.861067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.861043    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.861177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.861158    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.861221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.861161    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.862652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.862631    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.862785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.862769    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.868839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.868819    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:09.873816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.873789    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khszw\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.874014 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.873997    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:09.993397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:09.993363    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:10.156853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.156820    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"]
Oct 30 20:18:10.158578 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:10.158545    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680980fd_6738_4355_b88e_d413ed547214.slice/crio-ce43170cf6fdef0a01d6dccdf286165ca5193f1a582e1652336360ddd9075eba WatchSource:0}: Error finding container ce43170cf6fdef0a01d6dccdf286165ca5193f1a582e1652336360ddd9075eba: Status 404 returned error can't find the container with id ce43170cf6fdef0a01d6dccdf286165ca5193f1a582e1652336360ddd9075eba
Oct 30 20:18:10.160858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.160824    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:10.160960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.160903    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:10.160960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.160953    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:10.275681 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.275647    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"]
Oct 30 20:18:10.281048 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:10.281025    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074ea9fc_abf6_42a0_bd1c_3f472d66dfaf.slice/crio-181124f5e2337c32c5dd724096372d35c62bcf09d5efaebf9e3fe0a87151a24f WatchSource:0}: Error finding container 181124f5e2337c32c5dd724096372d35c62bcf09d5efaebf9e3fe0a87151a24f: Status 404 returned error can't find the container with id 181124f5e2337c32c5dd724096372d35c62bcf09d5efaebf9e3fe0a87151a24f
Oct 30 20:18:10.283658 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.283625    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:10.283782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.283705    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:10.283782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.283756    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:10.376124 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.376095    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:10.917577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.917542    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" event={"ID":"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf","Type":"ContainerStarted","Data":"3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d"}
Oct 30 20:18:10.917768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.917602    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" event={"ID":"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf","Type":"ContainerStarted","Data":"181124f5e2337c32c5dd724096372d35c62bcf09d5efaebf9e3fe0a87151a24f"}
Oct 30 20:18:10.919069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.919045    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" event={"ID":"680980fd-6738-4355-b88e-d413ed547214","Type":"ContainerStarted","Data":"054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c"}
Oct 30 20:18:10.919202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.919083    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" event={"ID":"680980fd-6738-4355-b88e-d413ed547214","Type":"ContainerStarted","Data":"ce43170cf6fdef0a01d6dccdf286165ca5193f1a582e1652336360ddd9075eba"}
Oct 30 20:18:10.993723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.993691    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:10.996389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.996331    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podStartSLOduration=1.996311161 podStartE2EDuration="1.996311161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:10.972179949 +0000 UTC m=+811.171752878" watchObservedRunningTime="2026-10-30 20:18:10.996311161 +0000 UTC m=+811.195884093"
Oct 30 20:18:10.996872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:10.996834    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podStartSLOduration=1.996824173 podStartE2EDuration="1.996824173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:10.995176824 +0000 UTC m=+811.194749753" watchObservedRunningTime="2026-10-30 20:18:10.996824173 +0000 UTC m=+811.196397102"
Oct 30 20:18:11.073708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.073669    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:11.303352 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.303312    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:11.869514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.869482    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:11.871845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.871809    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 20:18:11.871988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.871872    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 20:18:11.996002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.995972    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 20:18:11.996197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:11.996024    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 20:18:12.210058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:12.210020    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:12.871597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:12.871550    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 20:18:12.871983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:12.871640    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 20:18:12.994100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:12.994065    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 20:18:12.994282 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:12.994147    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 20:18:13.303774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:13.303724    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:13.870734 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:13.870683    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 20:18:13.870960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:13.870763    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 20:18:13.994747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:13.994713    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 20:18:13.995166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:13.994777    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 20:18:14.350621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:14.350546    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:14.871515 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:14.871476    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 20:18:14.872207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:14.871779    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 20:18:14.994452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:14.994414    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 20:18:14.994630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:14.994461    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 20:18:15.478563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:15.478525    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:15.874857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:15.874528    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 20:18:15.874857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:15.874592    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 20:18:15.994059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:15.994024    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 20:18:15.994239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:15.994082    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 20:18:16.606478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:16.606440    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:16.963402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:16.963307    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:16.963805 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:16.963777    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:16.965061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:16.965040    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:18:17.140592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:17.140536    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:17.141040 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:17.141020    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:17.141926 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:17.141903    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:18:17.729397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:17.729361    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:18.320331 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.320297    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"]
Oct 30 20:18:18.325551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.325531    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.331327 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.331303    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:18:18.331467 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.331364    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:18:18.333428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.333400    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"]
Oct 30 20:18:18.478520 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.478492    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.478684 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.478608    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbtmh\" (UniqueName: \"kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.478684 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.478642    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.573747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.573667    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:18.580331 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.580306    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.580477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.580399    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbtmh\" (UniqueName: \"kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.580477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.580425    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.581133 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.581095    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.582791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.582766    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.593322 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.593303    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbtmh\" (UniqueName: \"kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh\") pod \"maas-api-revoke-sub-f9660b20-ppqcz\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.662302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.662273    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:18.940370 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.940345    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"]
Oct 30 20:18:18.942375 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:18.942347    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fa1bf6_2042_4ef5_a549_d0066830bb13.slice/crio-2252e4e0b98eae199ccd4c4f6acdffd4587e6184b739b813829968f48018f86b WatchSource:0}: Error finding container 2252e4e0b98eae199ccd4c4f6acdffd4587e6184b739b813829968f48018f86b: Status 404 returned error can't find the container with id 2252e4e0b98eae199ccd4c4f6acdffd4587e6184b739b813829968f48018f86b
Oct 30 20:18:18.944399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:18.944383    6389 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:18:19.012793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:19.012761    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz" event={"ID":"c2fa1bf6-2042-4ef5-a549-d0066830bb13","Type":"ContainerStarted","Data":"2252e4e0b98eae199ccd4c4f6acdffd4587e6184b739b813829968f48018f86b"}
Oct 30 20:18:19.585351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:19.585320    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:20.130019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:20.129930    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:21.641249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:21.637674    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:23.017335 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:23.017292    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:24.085291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.085253    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:24.109606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.109574    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz" event={"ID":"c2fa1bf6-2042-4ef5-a549-d0066830bb13","Type":"ContainerStarted","Data":"f9f4f360b5c868d5fe0b21bce2cf200b094da309eff74fd6da6c9c52088c1971"}
Oct 30 20:18:24.215771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.215707    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz" podStartSLOduration=1.483830607 podStartE2EDuration="6.215688125s" totalImagesPullingTime="4.731857518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:24.136480869 +0000 UTC m=+824.336053797" watchObservedRunningTime="2026-10-30 20:18:24.215688125 +0000 UTC m=+824.415261055"
Oct 30 20:18:24.218166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.218138    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"]
Oct 30 20:18:24.222041 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.222024    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.234617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.234594    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-5197ec-openshift-default-dockercfg-fvmqh\""
Oct 30 20:18:24.237936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.237912    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"]
Oct 30 20:18:24.321591 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321555    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.321751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321607    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.321751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321718    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.321751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321743    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.321892 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321789    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.321930 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321914    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvbm\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.321970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321950    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.322045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.321996    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.322237 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.322211    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.423704 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423666    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.423896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423727    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.423896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423759    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.423896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423808    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.423896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423830    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.423896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423872    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423908    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvbm\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.423952    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.424006    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.424191    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424449 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.424429    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424794 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.424741    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424794 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.424767    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.424970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.424809    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.426278 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.426259    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.427209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.427189    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.438894 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.438790    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvbm\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.439884 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.439843    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.539366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.539336    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:24.827197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.827164    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"]
Oct 30 20:18:24.828441 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:24.828407    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1125734_af65_4eb1_a99b_887888ea4e6c.slice/crio-a0c0af6375f8bc9234723eee5d237eb61ea6c3850423725b330f4e9c17038dd8 WatchSource:0}: Error finding container a0c0af6375f8bc9234723eee5d237eb61ea6c3850423725b330f4e9c17038dd8: Status 404 returned error can't find the container with id a0c0af6375f8bc9234723eee5d237eb61ea6c3850423725b330f4e9c17038dd8
Oct 30 20:18:24.830900 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.830865    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:24.831029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.830946    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:24.831029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:24.830995    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:25.123468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.123387    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:25.124419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.124396    6389 generic.go:356] "Generic (PLEG): container finished" podID="c2fa1bf6-2042-4ef5-a549-d0066830bb13" containerID="f9f4f360b5c868d5fe0b21bce2cf200b094da309eff74fd6da6c9c52088c1971" exitCode=0
Oct 30 20:18:25.124542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.124451    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz" event={"ID":"c2fa1bf6-2042-4ef5-a549-d0066830bb13","Type":"ContainerDied","Data":"f9f4f360b5c868d5fe0b21bce2cf200b094da309eff74fd6da6c9c52088c1971"}
Oct 30 20:18:25.126134 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.126099    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" event={"ID":"c1125734-af65-4eb1-a99b-887888ea4e6c","Type":"ContainerStarted","Data":"b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3"}
Oct 30 20:18:25.126221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.126152    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" event={"ID":"c1125734-af65-4eb1-a99b-887888ea4e6c","Type":"ContainerStarted","Data":"a0c0af6375f8bc9234723eee5d237eb61ea6c3850423725b330f4e9c17038dd8"}
Oct 30 20:18:25.180430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.180377    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podStartSLOduration=1.180362109 podStartE2EDuration="1.180362109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:25.176207186 +0000 UTC m=+825.375780115" watchObservedRunningTime="2026-10-30 20:18:25.180362109 +0000 UTC m=+825.379935037"
Oct 30 20:18:25.542532 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.542501    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:25.547578 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.547545    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:18:25.547742 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.547598    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 20:18:25.799496 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:18:25.799410    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:18:25.964775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:25.964702    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:26.305247 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.305224    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:26.359169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.356404    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh\" (UniqueName: \"kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh\") pod \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") "
Oct 30 20:18:26.359169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.356475    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca\") pod \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") "
Oct 30 20:18:26.359169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.356625    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca\") pod \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\" (UID: \"c2fa1bf6-2042-4ef5-a549-d0066830bb13\") "
Oct 30 20:18:26.359169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.357150    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca" pod "c2fa1bf6-2042-4ef5-a549-d0066830bb13" (UID: "c2fa1bf6-2042-4ef5-a549-d0066830bb13"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:26.359169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.358791    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca" pod "c2fa1bf6-2042-4ef5-a549-d0066830bb13" (UID: "c2fa1bf6-2042-4ef5-a549-d0066830bb13"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:26.359169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.358999    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh" pod "c2fa1bf6-2042-4ef5-a549-d0066830bb13" (UID: "c2fa1bf6-2042-4ef5-a549-d0066830bb13"). InnerVolumeSpecName "kube-api-access-dbtmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:26.457897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.457812    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.457897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.457848    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbtmh\" (UniqueName: \"kubernetes.io/projected/c2fa1bf6-2042-4ef5-a549-d0066830bb13-kube-api-access-dbtmh\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.457897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.457866    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2fa1bf6-2042-4ef5-a549-d0066830bb13-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.546531 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.546496    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:18:26.546708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.546554    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 20:18:26.756637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:26.756601    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:27.153003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:27.152920    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"
Oct 30 20:18:27.153003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:27.152947    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz" event={"ID":"c2fa1bf6-2042-4ef5-a549-d0066830bb13","Type":"ContainerDied","Data":"2252e4e0b98eae199ccd4c4f6acdffd4587e6184b739b813829968f48018f86b"}
Oct 30 20:18:27.153003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:27.152996    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2252e4e0b98eae199ccd4c4f6acdffd4587e6184b739b813829968f48018f86b"
Oct 30 20:18:27.544142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:27.543495    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:18:27.544142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:27.543555    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 20:18:27.764494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:27.764460    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:28.534716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:28.534622    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:28.543472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:28.543443    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:18:28.543688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:28.543663    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 20:18:29.383573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:29.383543    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:29.548553 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:29.548518    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:18:29.548766 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:29.548744    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 20:18:30.169383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:30.169334    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:30.507330 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:30.507292    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:30.543143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:30.540588    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:18:30.543143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:30.540653    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 20:18:31.187962 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:31.187928    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:31.545051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:31.545021    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:31.545552 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:31.545322    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:31.546091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:31.546075    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:18:32.056917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:32.056877    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:33.090006 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:33.089968    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:33.537663 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:33.537625    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:35.034672 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:35.034632    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:36.305383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:36.305354    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:37.460526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:37.460491    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:38.237915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:38.237880    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:39.042991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:39.042953    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:39.993671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:39.993639    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"]
Oct 30 20:18:39.994145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:39.994113    6389 cpu_manager.go:414] "containerMap: removing container" podUID="c2fa1bf6-2042-4ef5-a549-d0066830bb13" containerName="revoke-keys"
Oct 30 20:18:39.994198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:39.994146    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2fa1bf6-2042-4ef5-a549-d0066830bb13" containerName="revoke-keys"
Oct 30 20:18:39.994250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:39.994241    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2fa1bf6-2042-4ef5-a549-d0066830bb13" containerName="revoke-keys"
Oct 30 20:18:39.997786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:39.997767    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.010205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.010179    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"]
Oct 30 20:18:40.029594 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.029572    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-75e734-openshift-default-dockercfg-dv849\""
Oct 30 20:18:40.053139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053088    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053145    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053367    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053418    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053493    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053517    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053610    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053639    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.053775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.053666    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmghc\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155044    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155080    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155096    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmghc\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155166    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155192    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155278    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155303    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155602 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155366    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155602 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155397    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155602 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155470    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155595    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155712    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.155829 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.155769    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.156037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.156019    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.157550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.157530    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.157782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.157762    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.168662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.168637    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.169069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.169051    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmghc\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.267322 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.267214    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:40.311646 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.311618    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:40.600425 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.600395    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"]
Oct 30 20:18:40.601237 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:40.601210    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbad6255_cd3b_49c6_bf1f_982304f50150.slice/crio-df6dc70dc5d6ff7ee2a62473db3bd8de8ae56204b34a78b57eaf691b9404dece WatchSource:0}: Error finding container df6dc70dc5d6ff7ee2a62473db3bd8de8ae56204b34a78b57eaf691b9404dece: Status 404 returned error can't find the container with id df6dc70dc5d6ff7ee2a62473db3bd8de8ae56204b34a78b57eaf691b9404dece
Oct 30 20:18:40.603849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.603820    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:40.603922 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.603883    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:40.603922 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.603913    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:40.990695 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:40.990664    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:41.303598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:41.303518    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" event={"ID":"fbad6255-cd3b-49c6-bf1f-982304f50150","Type":"ContainerStarted","Data":"11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312"}
Oct 30 20:18:41.303598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:41.303596    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" event={"ID":"fbad6255-cd3b-49c6-bf1f-982304f50150","Type":"ContainerStarted","Data":"df6dc70dc5d6ff7ee2a62473db3bd8de8ae56204b34a78b57eaf691b9404dece"}
Oct 30 20:18:41.311906 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:41.311872    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:41.313478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:41.313455    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:18:41.313585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:41.313499    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 20:18:41.872486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:41.872447    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:42.312501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:42.312470    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:18:42.312889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:42.312519    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 20:18:42.842659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:42.842620    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:43.312214 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:43.312179    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:18:43.312399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:43.312230    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 20:18:43.544587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:43.544552    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:44.295465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:44.295429    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:44.312426 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:44.312397    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:18:44.312573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:44.312448    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 20:18:45.160563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:45.160528    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:45.312106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:45.312067    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:18:45.312287 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:45.312146    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 20:18:45.984736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:45.984702    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:46.312154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:46.312045    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:18:46.312154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:46.312097    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 20:18:46.757207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:46.757165    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:47.317895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:47.317855    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:47.318364 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:47.318312    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:47.318971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:47.318950    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:18:47.346193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:47.346137    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podStartSLOduration=8.346111874 podStartE2EDuration="8.346111874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:41.341388989 +0000 UTC m=+841.540961952" watchObservedRunningTime="2026-10-30 20:18:47.346111874 +0000 UTC m=+847.545684867"
Oct 30 20:18:47.782697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:47.782662    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:47.894233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:47.894176    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:48.688782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:48.688742    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:49.553600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:49.553558    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:50.258061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.258026    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:50.916918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.916884    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"]
Oct 30 20:18:50.922197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.922167    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:50.932582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.932551    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-909519\"/\"distinct-2-w2-909519-kserve-self-signed-certs\""
Oct 30 20:18:50.932787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.932569    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"]
Oct 30 20:18:50.964732 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.964696    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:50.992875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.992842    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xjnm\" (UniqueName: \"kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:50.993061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.992887    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:50.993061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.992907    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:50.993223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.993079    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:50.993223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.993142    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:50.993331 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:50.993256    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.069305 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.069263    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:51.094813 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.094773    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xjnm\" (UniqueName: \"kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.094830    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.094854    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.094916    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.094936    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.094965    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.095383    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.095815    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.095990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.095977    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.098536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.098511    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.104377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.104350    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.108592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.108561    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xjnm\" (UniqueName: \"kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm\") pod \"distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") " pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.241743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.241712    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:51.527929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.527898    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"]
Oct 30 20:18:51.880947 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:51.880869    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:52.407002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:52.406967    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" event={"ID":"84969108-4995-4703-b73f-07edbff07dc5","Type":"ContainerStarted","Data":"c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d"}
Oct 30 20:18:52.407190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:52.407015    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" event={"ID":"84969108-4995-4703-b73f-07edbff07dc5","Type":"ContainerStarted","Data":"e4ec57d6468f76740d655db69979a4f2c71abbef4f992691032887db49b34c3a"}
Oct 30 20:18:52.432366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:52.432320    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podStartSLOduration=2.43230705 podStartE2EDuration="2.43230705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:52.429086146 +0000 UTC m=+852.628659084" watchObservedRunningTime="2026-10-30 20:18:52.43230705 +0000 UTC m=+852.631879979"
Oct 30 20:18:52.692836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:52.692723    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:53.242276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:53.242242    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:18:53.565324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:53.565251    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:54.460906 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.460869    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:54.615781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.615752    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"]
Oct 30 20:18:54.622701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.622676    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.640003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.639975    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"]
Oct 30 20:18:54.735262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735230    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735364    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735423    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735495    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735603    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735647    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735741 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735723    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735755    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.735855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.735835    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdkr9\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837237    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837294    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837332    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837357    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837408    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdkr9\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837494    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837585    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837615    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837639    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837832    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.837929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.837864    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.838055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.838010    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.838055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.838051    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.838549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.838519    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.839675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.839655    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.839924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.839903    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.850560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.850539    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdkr9\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.850640 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.850564    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.939398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.939370    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:54.954515 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.954486    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"]
Oct 30 20:18:54.959617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.959592    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:54.980402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:54.980370    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"]
Oct 30 20:18:55.142014 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.141983    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142048    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142103    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142315 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142274    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142334    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142369    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142419    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142500    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.142573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.142557    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcbt9\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.243413 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.243384    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"]
Oct 30 20:18:55.243877 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.243851    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.243980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.243907    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcbt9\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.243980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.243962    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244020    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244046    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244227 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244209    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244242    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244279    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244282    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244311    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244370    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244419    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244666    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.244959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.244926    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.245759 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:55.245731    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8f5a6d_cd2b_42b0_8020_5b805f498a5c.slice/crio-868ae13c3a07996314f9faa2e12398609799e705e33dc161f2a674ae2ac4cf1b WatchSource:0}: Error finding container 868ae13c3a07996314f9faa2e12398609799e705e33dc161f2a674ae2ac4cf1b: Status 404 returned error can't find the container with id 868ae13c3a07996314f9faa2e12398609799e705e33dc161f2a674ae2ac4cf1b
Oct 30 20:18:55.246803 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.246778    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.246936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.246916    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.248089 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.248064    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:55.248165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.248144    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:55.248207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.248175    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:55.259072 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.259052    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcbt9\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.259297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.259274    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.294060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.294036    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:55.374941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.374895    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:55.447039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.444750    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" event={"ID":"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c","Type":"ContainerStarted","Data":"7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903"}
Oct 30 20:18:55.447039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.444816    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" event={"ID":"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c","Type":"ContainerStarted","Data":"868ae13c3a07996314f9faa2e12398609799e705e33dc161f2a674ae2ac4cf1b"}
Oct 30 20:18:55.470860 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.470808    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podStartSLOduration=1.470791274 podStartE2EDuration="1.470791274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:55.469556074 +0000 UTC m=+855.669129003" watchObservedRunningTime="2026-10-30 20:18:55.470791274 +0000 UTC m=+855.670364202"
Oct 30 20:18:55.580692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.580668    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"]
Oct 30 20:18:55.581949 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:18:55.581910    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9eb0447_0ceb_4ec2_b9e9_b0d094885335.slice/crio-091c8df246147cb60c85bbf0c408e5a618982457db7b448c44727699e7d19171 WatchSource:0}: Error finding container 091c8df246147cb60c85bbf0c408e5a618982457db7b448c44727699e7d19171: Status 404 returned error can't find the container with id 091c8df246147cb60c85bbf0c408e5a618982457db7b448c44727699e7d19171
Oct 30 20:18:55.584388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.584355    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:55.584461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.584419    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:55.584461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.584446    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:18:55.793832 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:18:55.793788    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:18:55.939675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:55.939610    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:18:56.476698 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:56.476659    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" event={"ID":"d9eb0447-0ceb-4ec2-b9e9-b0d094885335","Type":"ContainerStarted","Data":"98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693"}
Oct 30 20:18:56.477318 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:56.477291    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" event={"ID":"d9eb0447-0ceb-4ec2-b9e9-b0d094885335","Type":"ContainerStarted","Data":"091c8df246147cb60c85bbf0c408e5a618982457db7b448c44727699e7d19171"}
Oct 30 20:18:56.503112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:56.503052    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podStartSLOduration=2.503025935 podStartE2EDuration="2.503025935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:56.500857132 +0000 UTC m=+856.700430062" watchObservedRunningTime="2026-10-30 20:18:56.503025935 +0000 UTC m=+856.702598935"
Oct 30 20:18:56.568142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:56.567646    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:56.944352 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:56.944262    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 20:18:56.944780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:56.944747    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 20:18:57.295259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:57.295229    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:18:57.296847 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:57.296824    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 20:18:57.296922 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:57.296870    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 20:18:57.564656 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:57.564527    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:57.944198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:57.943496    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 20:18:57.944198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:57.943556    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 20:18:58.295037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:58.294986    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 20:18:58.295233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:58.295047    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 20:18:58.667223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:58.666603    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:58.944198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:58.941736    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 20:18:58.944198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:58.941795    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 20:18:59.297655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:59.297571    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 20:18:59.297655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:59.297624    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 20:18:59.904208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:59.903833    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:18:59.945141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:59.944613    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 20:18:59.945141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:18:59.944669    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 20:19:00.300812 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:00.300692    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 20:19:00.300812 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:00.300753    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 20:19:00.945143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:00.941510    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 20:19:00.945143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:00.941567    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 20:19:01.245140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.244203    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:19:01.295156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.294540    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 20:19:01.295156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.294620    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 20:19:01.315285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.315218    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:19:01.450617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.450583    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:01.790942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.790639    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:19:01.940478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.940443    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 20:19:01.940676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:01.940514    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 20:19:02.294968 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:02.294933    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 20:19:02.295498 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:02.294996    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 20:19:02.959148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:02.958530    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:19:02.959325 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:02.959293    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:19:02.962018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:02.960575    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:19:03.155181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.155096    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:03.437875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.437840    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"]
Oct 30 20:19:03.445807 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.445771    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.456044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.453994    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-75e734\"/\"distinct-w6-75e734-kserve-self-signed-certs\""
Oct 30 20:19:03.457549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.457520    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"]
Oct 30 20:19:03.466655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.466630    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.466761 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.466670    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.466844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.466821    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbgmh\" (UniqueName: \"kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.466927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.466910    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.467051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.467033    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.467148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.467063    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.486708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.486676    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:03.567853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.567810    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.567858    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.567929    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.567956    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.568026    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbgmh\" (UniqueName: \"kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.568091    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568724 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.568699    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.568727    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.568991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.568931    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.572966 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.572936    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.573570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.573545    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.583822 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.583795    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbgmh\" (UniqueName: \"kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh\") pod \"distinct-w6-75e734-kserve-5688b6dc96-8gps8\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") " pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:03.771314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:03.771282    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:04.064941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.064908    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"]
Oct 30 20:19:04.065458 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:19:04.065432    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1295ab6_c34b_4fb8_acc5_89cc095986fc.slice/crio-acc33d6834f36e4d794a33a355ca96d325715df74621ccb94b5ad9d7e7e1d48b WatchSource:0}: Error finding container acc33d6834f36e4d794a33a355ca96d325715df74621ccb94b5ad9d7e7e1d48b: Status 404 returned error can't find the container with id acc33d6834f36e4d794a33a355ca96d325715df74621ccb94b5ad9d7e7e1d48b
Oct 30 20:19:04.336364 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.336276    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:19:04.336364 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.336343    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:19:04.511870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.511839    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:04.591717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.591616    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" event={"ID":"d1295ab6-c34b-4fb8-acc5-89cc095986fc","Type":"ContainerStarted","Data":"cfc26422c275c243022694e9fdbd6b0ab70c823e690e1a3680969c1c94577951"}
Oct 30 20:19:04.591717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.591677    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" event={"ID":"d1295ab6-c34b-4fb8-acc5-89cc095986fc","Type":"ContainerStarted","Data":"acc33d6834f36e4d794a33a355ca96d325715df74621ccb94b5ad9d7e7e1d48b"}
Oct 30 20:19:04.621579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.621524    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podStartSLOduration=1.621507861 podStartE2EDuration="1.621507861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:04.618916707 +0000 UTC m=+864.818489649" watchObservedRunningTime="2026-10-30 20:19:04.621507861 +0000 UTC m=+864.821080791"
Oct 30 20:19:04.736481 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.736450    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:19:04.737279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.737251    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:19:04.740332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.740305    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:19:04.772883 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:04.771952    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:05.327748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:05.327703    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:06.078948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:06.078911    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:06.981188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:06.981107    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:07.930854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:07.930825    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:08.768199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:08.768165    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:09.497981 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.497941    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:09.837174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.837055    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"]
Oct 30 20:19:09.842044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.842020    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.846820 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.846794    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-909519-openshift-default-dockercfg-r5l5q\""
Oct 30 20:19:09.857305 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.857278    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"]
Oct 30 20:19:09.952676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952644    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.952858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952694    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.952858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952720    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58ghl\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.952858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952830    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.952981 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952874    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.952981 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952895    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.952981 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.952930    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.953182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.953165    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:09.953287 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:09.953272    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054135    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054204    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054226    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054251    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054467 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054439    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054571    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054577    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054618    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054705    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054825 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054772    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054825 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054789    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054825 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054801    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58ghl\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054949 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054896    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.054949 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.054917    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.056921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.056892    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.057024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.057007    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.068163 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.068113    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.068459 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.068441    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58ghl\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.159501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.159421    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:10.275091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.275056    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:10.446011 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.445979    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"]
Oct 30 20:19:10.446554 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:19:10.446532    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc463c1ef_cffb_492d_94cc_61e42e93e99d.slice/crio-2be3827687f8226fd2d58ae273f785d292086a1c25cef452f83bda708e353011 WatchSource:0}: Error finding container 2be3827687f8226fd2d58ae273f785d292086a1c25cef452f83bda708e353011: Status 404 returned error can't find the container with id 2be3827687f8226fd2d58ae273f785d292086a1c25cef452f83bda708e353011
Oct 30 20:19:10.449060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.449029    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:10.449152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.449090    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:10.449201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.449169    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:10.632811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.632775    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" event={"ID":"c463c1ef-cffb-492d-94cc-61e42e93e99d","Type":"ContainerStarted","Data":"d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7"}
Oct 30 20:19:10.633384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.632827    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" event={"ID":"c463c1ef-cffb-492d-94cc-61e42e93e99d","Type":"ContainerStarted","Data":"2be3827687f8226fd2d58ae273f785d292086a1c25cef452f83bda708e353011"}
Oct 30 20:19:10.663524 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:10.663472    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podStartSLOduration=1.6634592719999999 podStartE2EDuration="1.663459272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:10.659984977 +0000 UTC m=+870.859557906" watchObservedRunningTime="2026-10-30 20:19:10.663459272 +0000 UTC m=+870.863032199"
Oct 30 20:19:11.160204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:11.160169    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:11.161714 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:11.161695    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:11.161772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:11.161740    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:11.379049 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:11.379016    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:12.160540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:12.160513    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:12.160791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:12.160565    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:12.274537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:12.274501    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:13.083620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:13.083588    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:13.159874 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:13.159838    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:13.160046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:13.159897    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:13.771606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:13.771571    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:13.784415 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:13.784392    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:14.086172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:14.086067    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:14.160568 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:14.160533    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:14.160775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:14.160587    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:14.685225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:14.685194    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:19:14.864396 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:14.864360    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:15.159959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:15.159921    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:15.160174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:15.159994    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:15.639565 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:15.639530    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:16.160049 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:16.160014    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:16.160486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:16.160068    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:16.674536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:16.674500    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:17.160742 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:17.160703    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:19:17.161150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:17.160783    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 20:19:17.669493 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:17.669454    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:18.788856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:18.788821    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:19.202308 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.202229    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:19:19.202308 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.202284    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:19:19.342385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.342348    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:19.342789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.342762    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:19.366042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.366016    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:19:19.657100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.657061    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:19.784370 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:19.784330    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:20.623132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:20.623084    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:21.355083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:21.355041    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:22.120059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:22.120022    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:22.992284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:22.992249    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:23.922436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:23.922400    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:24.758840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:24.758805    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:25.635764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:25.635723    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:25.799597 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:19:25.799554    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:19:26.602703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:26.602667    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:27.533394 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:27.533357    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:28.684908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:28.684875    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:29.633678 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:29.633642    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:30.461446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:30.461407    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:31.399990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:31.399953    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:32.245582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:32.245543    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:32.995718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:32.995684    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:33.608239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:33.608204    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:33.752055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:33.752027    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:34.667799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:34.667751    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:35.454829 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:35.454795    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:36.272811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:36.272777    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:37.002020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:37.001987    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:37.838083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:37.838040    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:38.567701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:38.567651    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:39.275865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:39.275835    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:40.032555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.032520    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:19:40.032821 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.032804    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:19:40.131860 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.131828    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"]
Oct 30 20:19:40.136701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.136675    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.186340 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.186307    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"]
Oct 30 20:19:40.259895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.259856    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260053 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.259913    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm22r\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260053 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260041    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260069    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260096    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260130    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260215    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260235    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.260374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.260276    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.325395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.325303    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:40.361437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361406    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361440    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361503    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361683 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361633    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361734 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361681    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm22r\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361734 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361726    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361838 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361751    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361838 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361805    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.361940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.361838    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.362160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.362105    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.362305 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.362166    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.362563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.362224    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.362824 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.362801    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.363037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.363004    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.365969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.365943    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.376766 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.376734    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.378215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.378186    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm22r\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.386015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.385986    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.453482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.453450    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:40.744809 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.744782    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"]
Oct 30 20:19:40.745661 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:19:40.745627    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode524ab0d_f1de_4473_bf97_30b7268c9a6c.slice/crio-54ce3a859d5dddf35db9c00c24164a549c2e25354a5d3b38167699258cd9db7b WatchSource:0}: Error finding container 54ce3a859d5dddf35db9c00c24164a549c2e25354a5d3b38167699258cd9db7b: Status 404 returned error can't find the container with id 54ce3a859d5dddf35db9c00c24164a549c2e25354a5d3b38167699258cd9db7b
Oct 30 20:19:40.749855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.749816    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:40.749962 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.749897    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:40.749962 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.749943    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:40.876134 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.876088    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" event={"ID":"e524ab0d-f1de-4473-bf97-30b7268c9a6c","Type":"ContainerStarted","Data":"f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49"}
Oct 30 20:19:40.876344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.876163    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" event={"ID":"e524ab0d-f1de-4473-bf97-30b7268c9a6c","Type":"ContainerStarted","Data":"54ce3a859d5dddf35db9c00c24164a549c2e25354a5d3b38167699258cd9db7b"}
Oct 30 20:19:40.902759 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:40.902710    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podStartSLOduration=0.902694613 podStartE2EDuration="902.694613ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:40.900492542 +0000 UTC m=+901.100065469" watchObservedRunningTime="2026-10-30 20:19:40.902694613 +0000 UTC m=+901.102267540"
Oct 30 20:19:41.164837 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:41.164754    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:41.455784 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:41.455690    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:41.457703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:41.457676    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:41.457844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:41.457734    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:42.175872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:42.175836    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:42.454668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:42.454584    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:42.454668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:42.454636    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:42.993805 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:42.993775    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:43.454660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.454571    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:43.454660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.454633    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:43.975019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.974988    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"]
Oct 30 20:19:43.978918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.978899    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:43.985534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.985493    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:19:43.985534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.985493    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:19:43.988576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:43.988551    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"]
Oct 30 20:19:44.049611 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.049571    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:44.106454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.106416    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2xn5\" (UniqueName: \"kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.106604 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.106563    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.106659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.106646    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.207946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.207915    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2xn5\" (UniqueName: \"kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.208193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.208010    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.208193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.208082    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.208768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.208741    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.210458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.210430    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.220610 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.220585    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2xn5\" (UniqueName: \"kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5\") pod \"maas-api-revoke-sub-b93e785e-rbf62\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.295313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.295225    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:44.454927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.454893    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:44.455087 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.454947    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:44.605066 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.605038    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"]
Oct 30 20:19:44.605975 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:19:44.605943    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f73dc17_1f22_4d03_8745_a639b449a755.slice/crio-2b4f54dff8ad0298459ad9af1b130165737c1dacc49eff9d77dea1829f045af6 WatchSource:0}: Error finding container 2b4f54dff8ad0298459ad9af1b130165737c1dacc49eff9d77dea1829f045af6: Status 404 returned error can't find the container with id 2b4f54dff8ad0298459ad9af1b130165737c1dacc49eff9d77dea1829f045af6
Oct 30 20:19:44.875605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.875557    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:44.903250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:44.903222    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62" event={"ID":"0f73dc17-1f22-4d03-8745-a639b449a755","Type":"ContainerStarted","Data":"2b4f54dff8ad0298459ad9af1b130165737c1dacc49eff9d77dea1829f045af6"}
Oct 30 20:19:45.454764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:45.454724    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:45.455199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:45.454790    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:45.789478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:45.789419    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:45.912421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:45.912389    6389 generic.go:356] "Generic (PLEG): container finished" podID="0f73dc17-1f22-4d03-8745-a639b449a755" containerID="611128eb9aff2b27364b9142657c85391bad46f7c0a91abf6b678f21965216a3" exitCode=0
Oct 30 20:19:45.920832 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:45.920796    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62" event={"ID":"0f73dc17-1f22-4d03-8745-a639b449a755","Type":"ContainerDied","Data":"611128eb9aff2b27364b9142657c85391bad46f7c0a91abf6b678f21965216a3"}
Oct 30 20:19:46.454605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:46.454562    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:46.454802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:46.454617    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:46.805442 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:46.805407    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:47.075354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.075327    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:47.163408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.163377    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca\") pod \"0f73dc17-1f22-4d03-8745-a639b449a755\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") "
Oct 30 20:19:47.163587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.163432    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca\") pod \"0f73dc17-1f22-4d03-8745-a639b449a755\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") "
Oct 30 20:19:47.163587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.163497    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5\" (UniqueName: \"kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5\") pod \"0f73dc17-1f22-4d03-8745-a639b449a755\" (UID: \"0f73dc17-1f22-4d03-8745-a639b449a755\") "
Oct 30 20:19:47.163740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.163716    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca" pod "0f73dc17-1f22-4d03-8745-a639b449a755" (UID: "0f73dc17-1f22-4d03-8745-a639b449a755"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:19:47.163872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.163855    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:19:47.165580 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.165548    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca" pod "0f73dc17-1f22-4d03-8745-a639b449a755" (UID: "0f73dc17-1f22-4d03-8745-a639b449a755"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:19:47.165704 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.165677    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5" pod "0f73dc17-1f22-4d03-8745-a639b449a755" (UID: "0f73dc17-1f22-4d03-8745-a639b449a755"). InnerVolumeSpecName "kube-api-access-f2xn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:19:47.265620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.265587    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0f73dc17-1f22-4d03-8745-a639b449a755-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:19:47.265620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.265615    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2xn5\" (UniqueName: \"kubernetes.io/projected/0f73dc17-1f22-4d03-8745-a639b449a755-kube-api-access-f2xn5\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:19:47.454455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.454370    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:19:47.454455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.454427    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 20:19:47.659574 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.659533    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:47.939064 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.939020    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62" event={"ID":"0f73dc17-1f22-4d03-8745-a639b449a755","Type":"ContainerDied","Data":"2b4f54dff8ad0298459ad9af1b130165737c1dacc49eff9d77dea1829f045af6"}
Oct 30 20:19:47.939238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.939153    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b4f54dff8ad0298459ad9af1b130165737c1dacc49eff9d77dea1829f045af6"
Oct 30 20:19:47.939403 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:47.939239    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"
Oct 30 20:19:48.457220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:48.457189    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:48.458170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:48.458150    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:48.458610 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:48.458591    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:48.459159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:48.459138    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:19:49.197024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:49.196987    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:50.068046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:50.068012    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:50.778576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:50.778545    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:51.671890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:51.671855    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:52.520401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:52.520373    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:53.570675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:53.570639    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:54.379759 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.379726    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:54.921166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.921131    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"]
Oct 30 20:19:54.921721 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.921706    6389 cpu_manager.go:414] "containerMap: removing container" podUID="0f73dc17-1f22-4d03-8745-a639b449a755" containerName="revoke-keys"
Oct 30 20:19:54.921764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.921722    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f73dc17-1f22-4d03-8745-a639b449a755" containerName="revoke-keys"
Oct 30 20:19:54.921843 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.921831    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f73dc17-1f22-4d03-8745-a639b449a755" containerName="revoke-keys"
Oct 30 20:19:54.925427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.925407    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:54.940054 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:54.940028    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"]
Oct 30 20:19:55.058928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.058898    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.058954    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059042    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059090    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njr25\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059171    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059195    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059293    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059381 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059326    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.059381 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.059340    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.160728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160691    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.160728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160734    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.160970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160806    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.160970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160851    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.160970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160878    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.160970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160934    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njr25\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.161316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.160984    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.161316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.161012    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.161316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.161091    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.161316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.161281    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.161559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.161354    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.161632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.161610    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.162018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.161926    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.162172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.162032    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.163256 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.163223    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:55.164268 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.164235    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.164566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.164539    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.171606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.171547    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"]
Oct 30 20:19:55.176824 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.176803    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.190271 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.190251    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.191036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.191014    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njr25\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.210943 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.210916    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"]
Oct 30 20:19:55.239793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.239772    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:55.262075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262047    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262085    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262292 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262273    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262303    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262336    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262510    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262565    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262669 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262643    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.262711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.262697    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps429\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363402    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363452    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363494    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363613    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363664    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363693    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363719    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps429\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363795    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.363918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.363826    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.364547 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.364502    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.366383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.364748    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.366383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.364935    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.366527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.365425    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.366527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.366162    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.368442 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.368419    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.371085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.371057    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.377535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.377510    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.378091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.378071    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps429\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429\") pod \"e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") " pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.528609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.528539    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:55.530351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.530323    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"]
Oct 30 20:19:55.531060 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:19:55.531035    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a40eda_7234_4bdc_b5f5_0509ad61aaa9.slice/crio-76dc9afe6885f16da293e8dcda84305fd9e982064f99b04cd8a4c3cca1bc786f WatchSource:0}: Error finding container 76dc9afe6885f16da293e8dcda84305fd9e982064f99b04cd8a4c3cca1bc786f: Status 404 returned error can't find the container with id 76dc9afe6885f16da293e8dcda84305fd9e982064f99b04cd8a4c3cca1bc786f
Oct 30 20:19:55.533547 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.533519    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:55.533631 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.533574    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:55.533631 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.533603    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:55.793528 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:19:55.793442    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:19:55.851219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.851194    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"]
Oct 30 20:19:55.852736 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:19:55.852704    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4ccf0b_c95d_426c_8670_1cce9d9c9699.slice/crio-2f37bcf891123d8f5bfbb29a0cc1dc6c60213997f98aed8a9ab6c76b08b5146a WatchSource:0}: Error finding container 2f37bcf891123d8f5bfbb29a0cc1dc6c60213997f98aed8a9ab6c76b08b5146a: Status 404 returned error can't find the container with id 2f37bcf891123d8f5bfbb29a0cc1dc6c60213997f98aed8a9ab6c76b08b5146a
Oct 30 20:19:55.855813 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.855771    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:55.855898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.855861    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:55.855937 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.855901    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:19:55.991741 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.991113    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" event={"ID":"af4ccf0b-c95d-426c-8670-1cce9d9c9699","Type":"ContainerStarted","Data":"60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5"}
Oct 30 20:19:55.991741 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.991190    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" event={"ID":"af4ccf0b-c95d-426c-8670-1cce9d9c9699","Type":"ContainerStarted","Data":"2f37bcf891123d8f5bfbb29a0cc1dc6c60213997f98aed8a9ab6c76b08b5146a"}
Oct 30 20:19:55.993105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.993073    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" event={"ID":"48a40eda-7234-4bdc-b5f5-0509ad61aaa9","Type":"ContainerStarted","Data":"b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178"}
Oct 30 20:19:55.993280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:55.993170    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" event={"ID":"48a40eda-7234-4bdc-b5f5-0509ad61aaa9","Type":"ContainerStarted","Data":"76dc9afe6885f16da293e8dcda84305fd9e982064f99b04cd8a4c3cca1bc786f"}
Oct 30 20:19:56.025194 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.025108    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podStartSLOduration=1.025086569 podStartE2EDuration="1.025086569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:56.018918302 +0000 UTC m=+916.218491229" watchObservedRunningTime="2026-10-30 20:19:56.025086569 +0000 UTC m=+916.224659497"
Oct 30 20:19:56.249141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.243714    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:19:56.249141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.243879    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:19:56.249141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.243925    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:19:56.259442 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.259410    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:56.546382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.546297    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:19:56.570732 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.570691    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:19:56.571027 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:56.570997    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:19:57.248270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:57.248232    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:19:57.248880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:57.248847    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:19:57.312634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:57.312590    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:57.533198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:57.529443    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:19:57.533198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:57.529515    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:19:58.253141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:58.252599    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:19:58.253141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:58.252674    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:19:58.542579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:58.542491    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:19:58.542825 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:58.542799    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:19:58.669790 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:58.669757    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:19:59.251756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:59.251721    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:19:59.252045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:59.252017    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:19:59.545199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:59.536501    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:19:59.545199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:59.536565    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:19:59.686349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:19:59.686321    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:00.242141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:00.241516    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:20:00.242141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:00.241581    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:20:00.543202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:00.529535    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:20:00.543202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:00.529598    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:20:00.674146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:00.663060    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:01.241140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:01.240553    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:20:01.241140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:01.240651    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:20:01.543202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:01.531518    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:20:01.543202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:01.531590    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:20:01.875404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:01.875307    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:02.240095 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:02.240062    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:20:02.240271 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:02.240136    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 20:20:02.529323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:02.529235    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 20:20:02.529323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:02.529305    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 20:20:02.757177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:02.757143    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:03.790908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:03.790858    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:20:03.792946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:03.792925    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:20:03.794713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:03.794697    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:20:03.830246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:03.830175    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podStartSLOduration=9.83009621 podStartE2EDuration="9.83009621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:56.054426689 +0000 UTC m=+916.253999620" watchObservedRunningTime="2026-10-30 20:20:03.83009621 +0000 UTC m=+924.029669139"
Oct 30 20:20:04.298140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:04.291227    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:04.298140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:04.291298    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.335138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.334212    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:05.335138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.334289    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.438255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.438205    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.513140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.512214    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.559280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.559235    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.568227 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.568188    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.624195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.623234    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.685224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.685176    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 20:20:05.685749 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.685185    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.790286 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.790250    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:05.790578 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.790548    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:05.806725 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.806681    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"]
Oct 30 20:20:05.822535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.822502    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:05.834988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.834958    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:20:05.843138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.842511    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:20:05.850139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.844777    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"]
Oct 30 20:20:05.946507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.946404    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:05.946791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.946750    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:05.947327 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:05.947301    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvdqp\" (UniqueName: \"kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.123852 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.123803    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvdqp\" (UniqueName: \"kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.124312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.124273    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.124483 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.124464    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.126503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.126480    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.225195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.219662    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.234849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.234818    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvdqp\" (UniqueName: \"kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp\") pod \"maas-api-revoke-sub-ed2a30c7-cn6gk\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.264429 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.264400    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:20:06.288141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.265956    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:20:06.320347 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.320317    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:06.351988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.351958    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:20:06.501504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.501429    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:06.859701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:06.859647    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"]
Oct 30 20:20:06.862614 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:20:06.862580    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5c740b_f7bd_47a3_8b25_afc30e35f6eb.slice/crio-3d0e81a5d3f9de46b83d8a03d8937774ef187a4b22ef6b17306f5dc79b3a4c72 WatchSource:0}: Error finding container 3d0e81a5d3f9de46b83d8a03d8937774ef187a4b22ef6b17306f5dc79b3a4c72: Status 404 returned error can't find the container with id 3d0e81a5d3f9de46b83d8a03d8937774ef187a4b22ef6b17306f5dc79b3a4c72
Oct 30 20:20:07.349725 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:07.349691    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk" event={"ID":"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb","Type":"ContainerStarted","Data":"a7380fb58fc122387e38da2aab4b6b4b490f1965631fc65cc4fe1b032826defe"}
Oct 30 20:20:07.349895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:07.349744    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk" event={"ID":"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb","Type":"ContainerStarted","Data":"3d0e81a5d3f9de46b83d8a03d8937774ef187a4b22ef6b17306f5dc79b3a4c72"}
Oct 30 20:20:07.373035 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:07.372983    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk" podStartSLOduration=2.372970503 podStartE2EDuration="2.372970503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:20:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:20:07.369874814 +0000 UTC m=+927.569447745" watchObservedRunningTime="2026-10-30 20:20:07.372970503 +0000 UTC m=+927.572543492"
Oct 30 20:20:08.358251 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:08.358216    6389 generic.go:356] "Generic (PLEG): container finished" podID="cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" containerID="a7380fb58fc122387e38da2aab4b6b4b490f1965631fc65cc4fe1b032826defe" exitCode=0
Oct 30 20:20:08.358499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:08.358328    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk" event={"ID":"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb","Type":"ContainerDied","Data":"a7380fb58fc122387e38da2aab4b6b4b490f1965631fc65cc4fe1b032826defe"}
Oct 30 20:20:08.548958 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:08.548930    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:09.516987 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.516963    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:09.570713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.570659    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:09.598789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.598758    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca\") pod \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") "
Oct 30 20:20:09.598953 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.598824    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca\") pod \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") "
Oct 30 20:20:09.598953 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.598861    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp\" (UniqueName: \"kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp\") pod \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\" (UID: \"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb\") "
Oct 30 20:20:09.599108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.599086    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca" pod "cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" (UID: "cc5c740b-f7bd-47a3-8b25-afc30e35f6eb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:20:09.599400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.599370    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:09.600982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.600948    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca" pod "cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" (UID: "cc5c740b-f7bd-47a3-8b25-afc30e35f6eb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:20:09.601102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.601082    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp" pod "cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" (UID: "cc5c740b-f7bd-47a3-8b25-afc30e35f6eb"). InnerVolumeSpecName "kube-api-access-tvdqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:20:09.701150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.701055    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:09.701150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:09.701085    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvdqp\" (UniqueName: \"kubernetes.io/projected/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb-kube-api-access-tvdqp\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:10.374300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:10.374264    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk" event={"ID":"cc5c740b-f7bd-47a3-8b25-afc30e35f6eb","Type":"ContainerDied","Data":"3d0e81a5d3f9de46b83d8a03d8937774ef187a4b22ef6b17306f5dc79b3a4c72"}
Oct 30 20:20:10.374300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:10.374291    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"
Oct 30 20:20:10.374300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:10.374312    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d0e81a5d3f9de46b83d8a03d8937774ef187a4b22ef6b17306f5dc79b3a4c72"
Oct 30 20:20:10.480305 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:10.480264    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:11.277428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:11.277394    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:12.114992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:12.114960    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:13.079008 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:13.078974    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:13.815174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:13.815105    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:14.639374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:14.639340    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:15.458342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:15.458304    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:16.421267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:16.421237    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:17.266492 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:17.266445    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:18.269840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.269801    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:18.944332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.944295    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"]
Oct 30 20:20:18.944871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.944857    6389 cpu_manager.go:414] "containerMap: removing container" podUID="cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" containerName="revoke-keys"
Oct 30 20:20:18.944921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.944872    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" containerName="revoke-keys"
Oct 30 20:20:18.944968 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.944959    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" containerName="revoke-keys"
Oct 30 20:20:18.949642 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.949621    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:18.955743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.955712    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:20:18.955909 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.955817    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:20:18.961648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.961616    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"]
Oct 30 20:20:18.987687 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:18.987648    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:19.018862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.018832    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.019059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.018985    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.019204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.019181    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx9fg\" (UniqueName: \"kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.120897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.120855    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.121092 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.120933    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.121092 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.121006    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx9fg\" (UniqueName: \"kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.121845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.121818    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.123307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.123279    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.134659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.134633    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx9fg\" (UniqueName: \"kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg\") pod \"maas-api-revoke-sub-e38850ba-hchfz\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.274538 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.274501    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:19.554541 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.554516    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"]
Oct 30 20:20:19.556082 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:20:19.556048    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9194bcce_31ad_4ce9_806e_0bb9f79866c1.slice/crio-7e99d7202e1f3d21b9593a92ef3457609f0ed6f7f3ab82141f4b6dae6c1cfad4 WatchSource:0}: Error finding container 7e99d7202e1f3d21b9593a92ef3457609f0ed6f7f3ab82141f4b6dae6c1cfad4: Status 404 returned error can't find the container with id 7e99d7202e1f3d21b9593a92ef3457609f0ed6f7f3ab82141f4b6dae6c1cfad4
Oct 30 20:20:19.705354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:19.705318    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:20.516692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:20.516657    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz" event={"ID":"9194bcce-31ad-4ce9-806e-0bb9f79866c1","Type":"ContainerStarted","Data":"eb3c035cf439295a3121597241e8d0f57a28d9897b52055d3ca7e8672d67c883"}
Oct 30 20:20:20.517308 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:20.517285    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz" event={"ID":"9194bcce-31ad-4ce9-806e-0bb9f79866c1","Type":"ContainerStarted","Data":"7e99d7202e1f3d21b9593a92ef3457609f0ed6f7f3ab82141f4b6dae6c1cfad4"}
Oct 30 20:20:20.587673 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:20.587608    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz" podStartSLOduration=2.5875843830000003 podStartE2EDuration="2.587584383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:20:20.566856454 +0000 UTC m=+940.766429383" watchObservedRunningTime="2026-10-30 20:20:20.587584383 +0000 UTC m=+940.787157311"
Oct 30 20:20:21.277142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:21.276305    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:21.528156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:21.528044    6389 generic.go:356] "Generic (PLEG): container finished" podID="9194bcce-31ad-4ce9-806e-0bb9f79866c1" containerID="eb3c035cf439295a3121597241e8d0f57a28d9897b52055d3ca7e8672d67c883" exitCode=0
Oct 30 20:20:21.528762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:21.528734    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz" event={"ID":"9194bcce-31ad-4ce9-806e-0bb9f79866c1","Type":"ContainerDied","Data":"eb3c035cf439295a3121597241e8d0f57a28d9897b52055d3ca7e8672d67c883"}
Oct 30 20:20:22.690943 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.690919    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:22.819084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.818984    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca\") pod \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") "
Oct 30 20:20:22.819084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.819041    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca\") pod \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") "
Oct 30 20:20:22.819340 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.819157    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg\" (UniqueName: \"kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg\") pod \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\" (UID: \"9194bcce-31ad-4ce9-806e-0bb9f79866c1\") "
Oct 30 20:20:22.819481 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.819451    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca" pod "9194bcce-31ad-4ce9-806e-0bb9f79866c1" (UID: "9194bcce-31ad-4ce9-806e-0bb9f79866c1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:20:22.819706 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.819681    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:22.821314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.821287    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca" pod "9194bcce-31ad-4ce9-806e-0bb9f79866c1" (UID: "9194bcce-31ad-4ce9-806e-0bb9f79866c1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:20:22.821667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.821638    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg" pod "9194bcce-31ad-4ce9-806e-0bb9f79866c1" (UID: "9194bcce-31ad-4ce9-806e-0bb9f79866c1"). InnerVolumeSpecName "kube-api-access-wx9fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:20:22.920300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.920271    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9194bcce-31ad-4ce9-806e-0bb9f79866c1-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:22.920300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:22.920295    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx9fg\" (UniqueName: \"kubernetes.io/projected/9194bcce-31ad-4ce9-806e-0bb9f79866c1-kube-api-access-wx9fg\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:23.077950 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:23.077863    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:23.542323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:23.542298    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"
Oct 30 20:20:23.542504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:23.542295    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz" event={"ID":"9194bcce-31ad-4ce9-806e-0bb9f79866c1","Type":"ContainerDied","Data":"7e99d7202e1f3d21b9593a92ef3457609f0ed6f7f3ab82141f4b6dae6c1cfad4"}
Oct 30 20:20:23.542504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:23.542429    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e99d7202e1f3d21b9593a92ef3457609f0ed6f7f3ab82141f4b6dae6c1cfad4"
Oct 30 20:20:24.013986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.013941    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:24.969370 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.969334    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"]
Oct 30 20:20:24.969945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.969921    6389 cpu_manager.go:414] "containerMap: removing container" podUID="9194bcce-31ad-4ce9-806e-0bb9f79866c1" containerName="revoke-keys"
Oct 30 20:20:24.969945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.969938    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9194bcce-31ad-4ce9-806e-0bb9f79866c1" containerName="revoke-keys"
Oct 30 20:20:24.970095 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.970017    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="9194bcce-31ad-4ce9-806e-0bb9f79866c1" containerName="revoke-keys"
Oct 30 20:20:24.973634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.973615    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:24.995915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:24.995879    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"]
Oct 30 20:20:25.019927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.019901    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:25.049330 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049298    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.049509 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049338    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.049509 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049375    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.049747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049723    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.050248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049765    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.050248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049922    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.050248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049943    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.050248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.049968    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c6zl\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.050248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.050096    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.076006 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.075968    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"]
Oct 30 20:20:25.080294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.080261    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.103392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.103341    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"]
Oct 30 20:20:25.151587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151555    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.151587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151590    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.151776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151611    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.151776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151687    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.151776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151756    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.151951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151777    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.151951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151794    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c6zl\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.151951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151841    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.151951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151864    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.151951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151908    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.151980    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152028    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152095    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2p9t\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.152226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152215    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152243    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152293    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152302    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152318    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152329    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152342    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.152460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152410    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.152756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152621    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.152756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.152679    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.154419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.154396    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.154582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.154562    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.165849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.165811    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c6zl\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.165972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.165899    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token\") pod \"e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") " pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.253712 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253648    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253712 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253696    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253714    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253742    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253756    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253773    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253802    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.253890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253882    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.254198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.253959    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2p9t\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.254198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.254071    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.254395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.254365    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.254506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.254411    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.254506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.254462    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.254582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.254555    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.256131 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.256098    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.256216 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.256107    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.266890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.266868    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.267104 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.267081    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2p9t\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t\") pod \"e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") " pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.287538 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.287513    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:25.408703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.408669    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:25.575861 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.575835    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"]
Oct 30 20:20:25.576763 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:20:25.576740    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79139b8_03e9_48f2_b257_3177cb352477.slice/crio-236c50c6db02fb2a6dd18ac300c0ea1734f74c019ce0bb9c02077dac6db7e85e WatchSource:0}: Error finding container 236c50c6db02fb2a6dd18ac300c0ea1734f74c019ce0bb9c02077dac6db7e85e: Status 404 returned error can't find the container with id 236c50c6db02fb2a6dd18ac300c0ea1734f74c019ce0bb9c02077dac6db7e85e
Oct 30 20:20:25.579363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.579324    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:20:25.579460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.579399    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:20:25.579460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.579435    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:20:25.696516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.696485    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"]
Oct 30 20:20:25.696824 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:20:25.696795    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e92526_b99e_4623_994c_56d089419dd6.slice/crio-8171a58af4decc40ebd779f8cd346f4976f7483bed1d93e56cdf7cd1165e849e WatchSource:0}: Error finding container 8171a58af4decc40ebd779f8cd346f4976f7483bed1d93e56cdf7cd1165e849e: Status 404 returned error can't find the container with id 8171a58af4decc40ebd779f8cd346f4976f7483bed1d93e56cdf7cd1165e849e
Oct 30 20:20:25.699555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.699520    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:20:25.699686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.699592    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:20:25.699686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.699636    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:20:25.807598 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:20:25.807523    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:20:25.869566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:25.869487    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:26.568516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.568419    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" event={"ID":"63e92526-b99e-4623-994c-56d089419dd6","Type":"ContainerStarted","Data":"187a52602b29a5c00425810c307c42111c800ed727716040bb4cf0eb9420abe4"}
Oct 30 20:20:26.568516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.568480    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" event={"ID":"63e92526-b99e-4623-994c-56d089419dd6","Type":"ContainerStarted","Data":"8171a58af4decc40ebd779f8cd346f4976f7483bed1d93e56cdf7cd1165e849e"}
Oct 30 20:20:26.570544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.570512    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" event={"ID":"b79139b8-03e9-48f2-b257-3177cb352477","Type":"ContainerStarted","Data":"78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6"}
Oct 30 20:20:26.570677 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.570558    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" event={"ID":"b79139b8-03e9-48f2-b257-3177cb352477","Type":"ContainerStarted","Data":"236c50c6db02fb2a6dd18ac300c0ea1734f74c019ce0bb9c02077dac6db7e85e"}
Oct 30 20:20:26.598391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.598330    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podStartSLOduration=1.598311794 podStartE2EDuration="1.598311794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:20:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:20:26.595193873 +0000 UTC m=+946.794766802" watchObservedRunningTime="2026-10-30 20:20:26.598311794 +0000 UTC m=+946.797884723"
Oct 30 20:20:26.625140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.621280    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podStartSLOduration=2.621261411 podStartE2EDuration="2.621261411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:20:26.620187826 +0000 UTC m=+946.819760755" watchObservedRunningTime="2026-10-30 20:20:26.621261411 +0000 UTC m=+946.820834343"
Oct 30 20:20:26.947201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:26.943435    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:27.287744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.287715    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:27.289235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.289208    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:27.289365 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.289259    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:27.409702 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.409668    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:27.411499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.411469    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:27.411610 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.411523    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:27.786456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:27.786395    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:28.289142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:28.289091    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:28.289341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:28.289174    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:28.409468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:28.409430    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:28.409641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:28.409493    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:28.914334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:28.914265    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:29.288691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:29.288655    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:29.288878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:29.288723    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:29.409369 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:29.409333    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:29.409544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:29.409392    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:29.890660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:29.890627    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:30.288589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:30.288553    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:30.289004 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:30.288605    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:30.409292 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:30.409258    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:30.409441 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:30.409314    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:31.288577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:31.288539    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:31.288910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:31.288882    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:31.416629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:31.416595    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:31.416933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:31.416889    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:31.547688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:31.547597    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:32.300142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.292026    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:32.300142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.292099    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:32.415143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.411565    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:32.415143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.411647    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:32.433678 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.433628    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 20:20:32.449066 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.449017    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:20:32.517257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.517207    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:32.543265 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.543214    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:32.579317 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.579211    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:32.678715 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.678670    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:20:32.698250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:32.698202    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:33.372222 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.372184    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:33.372828 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.372777    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:33.418144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.414474    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:33.418144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.414546    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.530759    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.531143    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.538810    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.558648    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.559220    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.559872    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.560180    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:20:33.619176 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.560227    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:20:33.642215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.621527    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 20:20:33.642215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.621949    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:20:33.650318 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.648192    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:33.712395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.687598    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:33.713028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.713001    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:20:33.849141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.784904    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:33.849141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.811218    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 20:20:33.849141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.811708    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:20:33.968201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:33.933245    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:34.124162 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.124097    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:34.124794 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.124758    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.299197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.293849    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:34.338885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.330330    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 20:20:34.338885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.330395    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 20:20:34.410633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.409569    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:34.412225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.411243    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:34.412225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.411295    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:34.416147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.415301    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:20:34.532064 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.531420    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:20:34.575600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.572270    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.575600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.572907    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:20:34.586019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.585997    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:20:34.598249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.596871    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:34.598249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.596932    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.601510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.601490    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:20:34.633775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.633733    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.658929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.657839    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.667148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.661571    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:20:34.713588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.713546    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.790052 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.789012    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:20:34.817259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.799525    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/health\": context deadline exceeded"
Oct 30 20:20:34.833239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.826462    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:20:34.844099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.843519    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 20:20:34.924144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.914306    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:34.924144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.914362    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:34.967439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:34.967331    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:20:35.206052 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.202167    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:20:35.232455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.228918    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:20:35.236294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.236270    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:20:35.242348 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.242328    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:20:35.278625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.278591    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:20:35.411208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.411177    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:20:35.411670 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:35.411230    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 20:20:36.289760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:36.289170    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:36.289760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:36.289242    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:37.321555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.321511    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:37.322053 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.321577    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:37.365700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.365656    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:37.366325 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.366301    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:37.367525 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.367504    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:20:37.450324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.450274    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:20:37.450527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.450348    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:37.583559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.583479    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:37.693160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.693104    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:37.693745 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.693720    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:37.694549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:37.694525    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:20:38.268757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.268718    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"]
Oct 30 20:20:38.274060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.274039    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.279260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.279238    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:20:38.279396 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.279304    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:20:38.283278 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.283257    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"]
Oct 30 20:20:38.378904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.378866    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.379402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.378949    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xxd7\" (UniqueName: \"kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.379402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.379040    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.480290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.480248    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.480466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.480318    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xxd7\" (UniqueName: \"kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.480466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.480350    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.481085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.481060    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.482679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.482658    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.493417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.493396    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xxd7\" (UniqueName: \"kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7\") pod \"maas-api-revoke-sub-d3244646-kcl4n\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.573109 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.573026    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:38.595466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.595439    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:38.899342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:38.899313    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"]
Oct 30 20:20:38.899861 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:20:38.899836    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf297cd7_0617_437d_9c8b_32b973bf6ca9.slice/crio-c686f7ce9390bb3f0c1f93aeac9155af836a0f34ac357f47680a20cec3e5fd3f WatchSource:0}: Error finding container c686f7ce9390bb3f0c1f93aeac9155af836a0f34ac357f47680a20cec3e5fd3f: Status 404 returned error can't find the container with id c686f7ce9390bb3f0c1f93aeac9155af836a0f34ac357f47680a20cec3e5fd3f
Oct 30 20:20:39.217067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:39.217034    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n" event={"ID":"cf297cd7-0617-437d-9c8b-32b973bf6ca9","Type":"ContainerStarted","Data":"c686f7ce9390bb3f0c1f93aeac9155af836a0f34ac357f47680a20cec3e5fd3f"}
Oct 30 20:20:39.590187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:39.590066    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:40.224730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:40.224693    6389 generic.go:356] "Generic (PLEG): container finished" podID="cf297cd7-0617-437d-9c8b-32b973bf6ca9" containerID="9c4e18918ff2455577640303f20b572d3c2dbd7a2f6dafa76a8a7f78920bf42c" exitCode=0
Oct 30 20:20:40.224909 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:40.224742    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n" event={"ID":"cf297cd7-0617-437d-9c8b-32b973bf6ca9","Type":"ContainerDied","Data":"9c4e18918ff2455577640303f20b572d3c2dbd7a2f6dafa76a8a7f78920bf42c"}
Oct 30 20:20:40.479838 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:40.479745    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:41.397927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.397897    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:41.425036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.425004    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:41.520350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.520259    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca\") pod \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") "
Oct 30 20:20:41.520350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.520308    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7\" (UniqueName: \"kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7\") pod \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") "
Oct 30 20:20:41.520601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.520485    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca\") pod \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\" (UID: \"cf297cd7-0617-437d-9c8b-32b973bf6ca9\") "
Oct 30 20:20:41.520850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.520825    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca" pod "cf297cd7-0617-437d-9c8b-32b973bf6ca9" (UID: "cf297cd7-0617-437d-9c8b-32b973bf6ca9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:20:41.521017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.520999    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:41.522421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.522397    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca" pod "cf297cd7-0617-437d-9c8b-32b973bf6ca9" (UID: "cf297cd7-0617-437d-9c8b-32b973bf6ca9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:20:41.522555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.522484    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7" pod "cf297cd7-0617-437d-9c8b-32b973bf6ca9" (UID: "cf297cd7-0617-437d-9c8b-32b973bf6ca9"). InnerVolumeSpecName "kube-api-access-7xxd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:20:41.622779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.622742    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf297cd7-0617-437d-9c8b-32b973bf6ca9-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:41.622779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:41.622775    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xxd7\" (UniqueName: \"kubernetes.io/projected/cf297cd7-0617-437d-9c8b-32b973bf6ca9-kube-api-access-7xxd7\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:42.240956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:42.240922    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"
Oct 30 20:20:42.241113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:42.240968    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n" event={"ID":"cf297cd7-0617-437d-9c8b-32b973bf6ca9","Type":"ContainerDied","Data":"c686f7ce9390bb3f0c1f93aeac9155af836a0f34ac357f47680a20cec3e5fd3f"}
Oct 30 20:20:42.241113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:42.241031    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c686f7ce9390bb3f0c1f93aeac9155af836a0f34ac357f47680a20cec3e5fd3f"
Oct 30 20:20:42.259592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:42.259565    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:43.032064 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:43.032025    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:43.826680 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:43.826650    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:45.618350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:45.618305    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:45.639257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:45.639210    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:45.724257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:45.724210    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.247294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.247246    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.284142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.281154    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.427275 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.427221    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.443142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.443006    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:46.472277 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.472233    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.538201 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.537382    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.620773 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.620226    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:46.623225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.621784    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:20:46.623225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.622099    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:20:46.761633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.756753    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:20:47.057001 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.779385    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": context deadline exceeded"
Oct 30 20:20:47.057339 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:46.844541    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.078593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.072340    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:20:47.282323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.282257    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.343270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.320141    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:20:47.474150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.448804    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.475802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.474427    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.476436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.476414    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:20:47.476616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.476602    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:20:47.584143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.582236    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.584143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.582822    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:20:47.624192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.617916    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.629795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.629753    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.641353 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.641304    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:20:47.668162 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.642015    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:20:47.696482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.677341    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:20:47.897671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:47.895257    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:20:48.080683 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.067784    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:20:48.207465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.207373    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:20:48.207748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.207727    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:20:48.207921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.207908    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:20:48.208051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.208039    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:20:48.208228 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.208213    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:20:48.208377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:48.208364    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:20:49.832022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.831991    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"]
Oct 30 20:20:49.832556 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.832541    6389 cpu_manager.go:414] "containerMap: removing container" podUID="cf297cd7-0617-437d-9c8b-32b973bf6ca9" containerName="revoke-keys"
Oct 30 20:20:49.832556 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.832555    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf297cd7-0617-437d-9c8b-32b973bf6ca9" containerName="revoke-keys"
Oct 30 20:20:49.832685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.832677    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf297cd7-0617-437d-9c8b-32b973bf6ca9" containerName="revoke-keys"
Oct 30 20:20:49.836145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.836110    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.842755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.842731    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:20:49.842919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.842832    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:20:49.847565 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.847543    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"]
Oct 30 20:20:49.893966 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.893930    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6prjg\" (UniqueName: \"kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.894141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.894000    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.894250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.894226    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.995853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.995814    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.996048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.995877    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6prjg\" (UniqueName: \"kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.996048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.995936    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.996700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.996671    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:49.998379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:49.998358    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:50.010398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:50.010369    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6prjg\" (UniqueName: \"kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg\") pod \"maas-api-revoke-sub-2ffe1807-lxbk4\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:50.167986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:50.167906    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:50.209776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:50.209732    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:50.453750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:50.453702    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"]
Oct 30 20:20:50.455194 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:20:50.455163    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd6c549_e68a_437c_9310_d28abca48949.slice/crio-b2a7384058e2a94deba2a7b97b2cf1b819049df6181f8fc770561303cbd86a40 WatchSource:0}: Error finding container b2a7384058e2a94deba2a7b97b2cf1b819049df6181f8fc770561303cbd86a40: Status 404 returned error can't find the container with id b2a7384058e2a94deba2a7b97b2cf1b819049df6181f8fc770561303cbd86a40
Oct 30 20:20:51.088975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:51.088895    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:51.470790 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:51.470762    6389 generic.go:356] "Generic (PLEG): container finished" podID="2fd6c549-e68a-437c-9310-d28abca48949" containerID="30694bf0fb8f1b3d51dd46d82318425caed5495c3e65a285ebb26c53a147abbf" exitCode=0
Oct 30 20:20:51.470980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:51.470809    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4" event={"ID":"2fd6c549-e68a-437c-9310-d28abca48949","Type":"ContainerDied","Data":"30694bf0fb8f1b3d51dd46d82318425caed5495c3e65a285ebb26c53a147abbf"}
Oct 30 20:20:51.470980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:51.470852    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4" event={"ID":"2fd6c549-e68a-437c-9310-d28abca48949","Type":"ContainerStarted","Data":"b2a7384058e2a94deba2a7b97b2cf1b819049df6181f8fc770561303cbd86a40"}
Oct 30 20:20:51.940770 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:51.940700    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:52.632747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.632723    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:52.729996 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.729922    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca\") pod \"2fd6c549-e68a-437c-9310-d28abca48949\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") "
Oct 30 20:20:52.729996 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.729964    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca\") pod \"2fd6c549-e68a-437c-9310-d28abca48949\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") "
Oct 30 20:20:52.730267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.730027    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg\" (UniqueName: \"kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg\") pod \"2fd6c549-e68a-437c-9310-d28abca48949\" (UID: \"2fd6c549-e68a-437c-9310-d28abca48949\") "
Oct 30 20:20:52.730372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.730348    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca" pod "2fd6c549-e68a-437c-9310-d28abca48949" (UID: "2fd6c549-e68a-437c-9310-d28abca48949"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:20:52.730492 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.730477    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:52.732063 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.732040    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca" pod "2fd6c549-e68a-437c-9310-d28abca48949" (UID: "2fd6c549-e68a-437c-9310-d28abca48949"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:20:52.732173 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.732129    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg" pod "2fd6c549-e68a-437c-9310-d28abca48949" (UID: "2fd6c549-e68a-437c-9310-d28abca48949"). InnerVolumeSpecName "kube-api-access-6prjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:20:52.831942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.831909    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fd6c549-e68a-437c-9310-d28abca48949-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:52.831942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.831937    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6prjg\" (UniqueName: \"kubernetes.io/projected/2fd6c549-e68a-437c-9310-d28abca48949-kube-api-access-6prjg\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:52.862816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:52.862783    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:53.486048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:53.486015    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4" event={"ID":"2fd6c549-e68a-437c-9310-d28abca48949","Type":"ContainerDied","Data":"b2a7384058e2a94deba2a7b97b2cf1b819049df6181f8fc770561303cbd86a40"}
Oct 30 20:20:53.486236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:53.486065    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2a7384058e2a94deba2a7b97b2cf1b819049df6181f8fc770561303cbd86a40"
Oct 30 20:20:53.486236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:53.486031    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"
Oct 30 20:20:53.658901 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:53.658865    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:54.552468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:54.552412    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:55.288205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:55.288166    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:55.794476 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:20:55.794430    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:20:56.030332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:56.030291    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:56.884577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:56.884536    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:57.580403 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:57.580367    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:58.433673 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:58.433629    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:59.175865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:59.175830    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:20:59.940555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:20:59.940511    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:00.775239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:00.775207    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:02.489660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:02.489621    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:02.842232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:02.841375    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:03.056143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:03.041254    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:03.068152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:03.065408    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:03.184461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:03.183206    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:03.661142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:03.651175    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:03.843141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:03.842217    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:03.843141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:03.842933    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:21:04.068505 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.067501    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:04.071596 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.071260    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:04.104309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.104216    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:04.219613 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.219582    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:21:04.220208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.220177    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:21:04.220332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.220235    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:21:04.220332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.220269    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:21:04.220332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:04.220298    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:21:05.193546 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:05.193501    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:06.365171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:06.365137    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:07.229624 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.229578    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:07.958558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.958526    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"]
Oct 30 20:21:07.959099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.959085    6389 cpu_manager.go:414] "containerMap: removing container" podUID="2fd6c549-e68a-437c-9310-d28abca48949" containerName="revoke-keys"
Oct 30 20:21:07.959179 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.959100    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fd6c549-e68a-437c-9310-d28abca48949" containerName="revoke-keys"
Oct 30 20:21:07.959217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.959194    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fd6c549-e68a-437c-9310-d28abca48949" containerName="revoke-keys"
Oct 30 20:21:07.962744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.962723    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:07.963099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.963070    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:07.973071 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.973047    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:21:07.973597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.973178    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:21:07.973921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:07.973891    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"]
Oct 30 20:21:08.034069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.034016    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.034342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.034316    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.034432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.034378    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c928\" (UniqueName: \"kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.135852 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.135815    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.136002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.135864    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c928\" (UniqueName: \"kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.136002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.135907    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.136570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.136550    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.138247 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.138226    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.149740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.149712    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c928\" (UniqueName: \"kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928\") pod \"maas-api-revoke-sub-be1f16a5-5jq96\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.317942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.317907    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:08.601321 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.601290    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"]
Oct 30 20:21:08.601757 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:08.601732    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17cb30fb_a840_44f7_a58f_7255fb5133fe.slice/crio-05c30409269cac0f57a7ebc95d8452d4e5940bc736782e1fd0ac32bf3cfdb1f7 WatchSource:0}: Error finding container 05c30409269cac0f57a7ebc95d8452d4e5940bc736782e1fd0ac32bf3cfdb1f7: Status 404 returned error can't find the container with id 05c30409269cac0f57a7ebc95d8452d4e5940bc736782e1fd0ac32bf3cfdb1f7
Oct 30 20:21:08.740468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.740431    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96" event={"ID":"17cb30fb-a840-44f7-a58f-7255fb5133fe","Type":"ContainerStarted","Data":"05c30409269cac0f57a7ebc95d8452d4e5940bc736782e1fd0ac32bf3cfdb1f7"}
Oct 30 20:21:08.861782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:08.861709    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:09.568348 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:09.568313    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:09.748614 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:09.748584    6389 generic.go:356] "Generic (PLEG): container finished" podID="17cb30fb-a840-44f7-a58f-7255fb5133fe" containerID="0db957b49549c2e5a18865e497714a876a346f6f25ae24483d454e6bff234f09" exitCode=0
Oct 30 20:21:09.748795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:09.748663    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96" event={"ID":"17cb30fb-a840-44f7-a58f-7255fb5133fe","Type":"ContainerDied","Data":"0db957b49549c2e5a18865e497714a876a346f6f25ae24483d454e6bff234f09"}
Oct 30 20:21:10.297363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.297327    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:10.894542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.894519    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:10.972211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.972177    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca\") pod \"17cb30fb-a840-44f7-a58f-7255fb5133fe\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") "
Oct 30 20:21:10.972395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.972231    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928\" (UniqueName: \"kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928\") pod \"17cb30fb-a840-44f7-a58f-7255fb5133fe\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") "
Oct 30 20:21:10.972395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.972329    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca\") pod \"17cb30fb-a840-44f7-a58f-7255fb5133fe\" (UID: \"17cb30fb-a840-44f7-a58f-7255fb5133fe\") "
Oct 30 20:21:10.972527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.972502    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca" pod "17cb30fb-a840-44f7-a58f-7255fb5133fe" (UID: "17cb30fb-a840-44f7-a58f-7255fb5133fe"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:10.972846 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.972825    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:10.974320 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.974294    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca" pod "17cb30fb-a840-44f7-a58f-7255fb5133fe" (UID: "17cb30fb-a840-44f7-a58f-7255fb5133fe"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:10.974421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:10.974403    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928" pod "17cb30fb-a840-44f7-a58f-7255fb5133fe" (UID: "17cb30fb-a840-44f7-a58f-7255fb5133fe"). InnerVolumeSpecName "kube-api-access-7c928". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:11.073834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.073756    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c928\" (UniqueName: \"kubernetes.io/projected/17cb30fb-a840-44f7-a58f-7255fb5133fe-kube-api-access-7c928\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:11.073834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.073787    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17cb30fb-a840-44f7-a58f-7255fb5133fe-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:11.179542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.179498    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:11.762628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.762598    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"
Oct 30 20:21:11.762798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.762632    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96" event={"ID":"17cb30fb-a840-44f7-a58f-7255fb5133fe","Type":"ContainerDied","Data":"05c30409269cac0f57a7ebc95d8452d4e5940bc736782e1fd0ac32bf3cfdb1f7"}
Oct 30 20:21:11.762798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.762682    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05c30409269cac0f57a7ebc95d8452d4e5940bc736782e1fd0ac32bf3cfdb1f7"
Oct 30 20:21:11.915420 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:11.915384    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:12.660505 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:12.660471    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:13.648752 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:13.648721    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:14.506261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:14.506224    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:15.310386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:15.310331    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:16.121339 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:16.121309    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:16.965394 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:16.965356    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:19.041112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:19.041042    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:19.058468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:19.058430    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:21:19.071024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:19.070985    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:19.514972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:19.514931    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:21:19.515301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:19.515266    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:19.997894 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:19.997859    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:20.808676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.808638    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"]
Oct 30 20:21:20.809212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.809197    6389 cpu_manager.go:414] "containerMap: removing container" podUID="17cb30fb-a840-44f7-a58f-7255fb5133fe" containerName="revoke-keys"
Oct 30 20:21:20.809264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.809213    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17cb30fb-a840-44f7-a58f-7255fb5133fe" containerName="revoke-keys"
Oct 30 20:21:20.809376 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.809365    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="17cb30fb-a840-44f7-a58f-7255fb5133fe" containerName="revoke-keys"
Oct 30 20:21:20.813053 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.813037    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:20.818798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.818761    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:21:20.818970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.818946    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:21:20.821611 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.821588    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"]
Oct 30 20:21:20.939752 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.939717    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:20.939941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.939801    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69zlg\" (UniqueName: \"kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:20.939941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:20.939838    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.041432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.041236    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.041432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.041289    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69zlg\" (UniqueName: \"kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.041432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.041331    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.043068 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.043042    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.044946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.044922    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.061388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.059266    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69zlg\" (UniqueName: \"kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg\") pod \"maas-api-revoke-sub-771980ae-h9hzd\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.128735 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.128702    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:21.440261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.440228    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"]
Oct 30 20:21:21.441424 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:21.441398    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e50323_5c6e_469a_a2cc_61fb34920f58.slice/crio-d3a92950ad14333aa8be1d803db34201149714b9ca5e3f96c13e8b5d21fbd2d2 WatchSource:0}: Error finding container d3a92950ad14333aa8be1d803db34201149714b9ca5e3f96c13e8b5d21fbd2d2: Status 404 returned error can't find the container with id d3a92950ad14333aa8be1d803db34201149714b9ca5e3f96c13e8b5d21fbd2d2
Oct 30 20:21:21.684192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.684155    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:21.816104 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.816024    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"]
Oct 30 20:21:21.823939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.823912    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.829895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.829871    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"]
Oct 30 20:21:21.860621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.860584    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.860771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.860668    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s84gk\" (UniqueName: \"kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.860970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.860947    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.901434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.901400    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd" event={"ID":"d9e50323-5c6e-469a-a2cc-61fb34920f58","Type":"ContainerStarted","Data":"79d22941155679f167126959f9711d0796ca41533ce7daa5d71772c95884a3cd"}
Oct 30 20:21:21.901588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.901463    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd" event={"ID":"d9e50323-5c6e-469a-a2cc-61fb34920f58","Type":"ContainerStarted","Data":"d3a92950ad14333aa8be1d803db34201149714b9ca5e3f96c13e8b5d21fbd2d2"}
Oct 30 20:21:21.926042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.925971    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd" podStartSLOduration=1.925952842 podStartE2EDuration="1.925952842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:21:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:21:21.922177366 +0000 UTC m=+1002.121750295" watchObservedRunningTime="2026-10-30 20:21:21.925952842 +0000 UTC m=+1002.125525772"
Oct 30 20:21:21.962411 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.962379    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.962564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.962431    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.962564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.962473    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s84gk\" (UniqueName: \"kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.963252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.963228    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.965083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.965059    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:21.975257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:21.975235    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s84gk\" (UniqueName: \"kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk\") pod \"maas-api-revoke-sub-29346fc3-7wts2\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:22.154809 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:22.154737    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:22.438512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:22.438482    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"]
Oct 30 20:21:22.439485 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:22.439461    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f77e30_4745_4424_b7e3_76c2bbb1ea36.slice/crio-0c811e247f1139830a6bafda6fb7e323dc867248c7877b2bb94b6d9871ff8057 WatchSource:0}: Error finding container 0c811e247f1139830a6bafda6fb7e323dc867248c7877b2bb94b6d9871ff8057: Status 404 returned error can't find the container with id 0c811e247f1139830a6bafda6fb7e323dc867248c7877b2bb94b6d9871ff8057
Oct 30 20:21:22.686654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:22.686564    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:22.910904 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:22.910868    6389 generic.go:356] "Generic (PLEG): container finished" podID="d9e50323-5c6e-469a-a2cc-61fb34920f58" containerID="79d22941155679f167126959f9711d0796ca41533ce7daa5d71772c95884a3cd" exitCode=0
Oct 30 20:21:22.911396 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:22.910948    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd" event={"ID":"d9e50323-5c6e-469a-a2cc-61fb34920f58","Type":"ContainerDied","Data":"79d22941155679f167126959f9711d0796ca41533ce7daa5d71772c95884a3cd"}
Oct 30 20:21:22.912378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:22.912356    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2" event={"ID":"a0f77e30-4745-4424-b7e3-76c2bbb1ea36","Type":"ContainerStarted","Data":"0c811e247f1139830a6bafda6fb7e323dc867248c7877b2bb94b6d9871ff8057"}
Oct 30 20:21:23.432561 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:23.432528    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:23.919638 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:23.919607    6389 generic.go:356] "Generic (PLEG): container finished" podID="a0f77e30-4745-4424-b7e3-76c2bbb1ea36" containerID="b2c285949b1420970d443d480af89e1bc8e087f4ba47740c16ebcfda5da7358d" exitCode=0
Oct 30 20:21:23.920028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:23.919693    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2" event={"ID":"a0f77e30-4745-4424-b7e3-76c2bbb1ea36","Type":"ContainerDied","Data":"b2c285949b1420970d443d480af89e1bc8e087f4ba47740c16ebcfda5da7358d"}
Oct 30 20:21:24.078646 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.078617    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:24.188819 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.188741    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg\" (UniqueName: \"kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg\") pod \"d9e50323-5c6e-469a-a2cc-61fb34920f58\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") "
Oct 30 20:21:24.188819 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.188787    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca\") pod \"d9e50323-5c6e-469a-a2cc-61fb34920f58\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") "
Oct 30 20:21:24.189053 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.189013    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca\") pod \"d9e50323-5c6e-469a-a2cc-61fb34920f58\" (UID: \"d9e50323-5c6e-469a-a2cc-61fb34920f58\") "
Oct 30 20:21:24.189420 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.189391    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca" pod "d9e50323-5c6e-469a-a2cc-61fb34920f58" (UID: "d9e50323-5c6e-469a-a2cc-61fb34920f58"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:24.189626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.189605    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:24.190782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.190748    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg" pod "d9e50323-5c6e-469a-a2cc-61fb34920f58" (UID: "d9e50323-5c6e-469a-a2cc-61fb34920f58"). InnerVolumeSpecName "kube-api-access-69zlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:24.190883 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.190834    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca" pod "d9e50323-5c6e-469a-a2cc-61fb34920f58" (UID: "d9e50323-5c6e-469a-a2cc-61fb34920f58"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:24.291224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.291200    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69zlg\" (UniqueName: \"kubernetes.io/projected/d9e50323-5c6e-469a-a2cc-61fb34920f58-kube-api-access-69zlg\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:24.291224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.291225    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9e50323-5c6e-469a-a2cc-61fb34920f58-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:24.472490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.472458    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:24.719724 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.719689    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:24.927477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.927401    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"
Oct 30 20:21:24.927477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.927419    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd" event={"ID":"d9e50323-5c6e-469a-a2cc-61fb34920f58","Type":"ContainerDied","Data":"d3a92950ad14333aa8be1d803db34201149714b9ca5e3f96c13e8b5d21fbd2d2"}
Oct 30 20:21:24.927477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:24.927464    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3a92950ad14333aa8be1d803db34201149714b9ca5e3f96c13e8b5d21fbd2d2"
Oct 30 20:21:25.091759 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.091736    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:25.201450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.201372    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca\") pod \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") "
Oct 30 20:21:25.201592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.201457    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca\") pod \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") "
Oct 30 20:21:25.201743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.201712    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk\" (UniqueName: \"kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk\") pod \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\" (UID: \"a0f77e30-4745-4424-b7e3-76c2bbb1ea36\") "
Oct 30 20:21:25.201892 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.201768    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca" pod "a0f77e30-4745-4424-b7e3-76c2bbb1ea36" (UID: "a0f77e30-4745-4424-b7e3-76c2bbb1ea36"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:25.202190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.202170    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:25.203634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.203612    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk" pod "a0f77e30-4745-4424-b7e3-76c2bbb1ea36" (UID: "a0f77e30-4745-4424-b7e3-76c2bbb1ea36"). InnerVolumeSpecName "kube-api-access-s84gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:25.203736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.203613    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca" pod "a0f77e30-4745-4424-b7e3-76c2bbb1ea36" (UID: "a0f77e30-4745-4424-b7e3-76c2bbb1ea36"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:25.303608 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.303579    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s84gk\" (UniqueName: \"kubernetes.io/projected/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-kube-api-access-s84gk\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:25.303608 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.303606    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0f77e30-4745-4424-b7e3-76c2bbb1ea36-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:25.317231 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.317198    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:25.793067 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:21:25.793032    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:21:25.935091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.935064    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"
Oct 30 20:21:25.935460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.935089    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2" event={"ID":"a0f77e30-4745-4424-b7e3-76c2bbb1ea36","Type":"ContainerDied","Data":"0c811e247f1139830a6bafda6fb7e323dc867248c7877b2bb94b6d9871ff8057"}
Oct 30 20:21:25.935460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:25.935173    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c811e247f1139830a6bafda6fb7e323dc867248c7877b2bb94b6d9871ff8057"
Oct 30 20:21:26.060354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:26.060248    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:26.910536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:26.910503    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:27.658611 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:27.658576    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:28.389327 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:28.389297    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:29.276408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:29.276376    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:31.517589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:31.517556    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:31.638756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:31.638705    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:21:31.960277 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:31.954180    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.056144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.051667    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.080152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.056415    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.080152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.070303    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.080152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.074515    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.129300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.129254    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.182257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.182207    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.248142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.244201    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.248142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.245340    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.647190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.645230    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.647190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.645994    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:21:32.651140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.647821    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:21:32.958293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.958192    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:32.959227 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:32.959201    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:21:33.077314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.077268    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:33.114143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.087882    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:21:33.123144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.093731    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:21:33.123144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.117563    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:21:33.179158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.171149    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:21:33.179158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.171669    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:21:33.225063 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.224236    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:33.225063 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.224665    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:21:33.260161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.231714    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:33.265400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.253921    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:33.273443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.255204    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:33.277816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.264017    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:21:33.287147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.278025    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:21:33.287344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.287326    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:21:33.390560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.390521    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:21:33.390827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.390795    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:33.648199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.646543    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:21:33.660855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.660235    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 20:21:33.661709 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.661108    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:21:33.753529 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:33.752528    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:21:34.077255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.077223    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:21:34.077532 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.077519    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:21:34.077675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.077665    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:21:34.090164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.090127    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:21:34.170787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.169882    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:21:34.171558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.171521    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:34.172358 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.172297    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:21:34.178683 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.178664    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:21:34.179705 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.179683    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:21:34.179811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.179726    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:21:34.219389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.219348    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:21:34.584550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:34.584520    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:36.591606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:36.591571    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:37.371479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.371448    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"]
Oct 30 20:21:37.372054 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.372034    6389 cpu_manager.go:414] "containerMap: removing container" podUID="a0f77e30-4745-4424-b7e3-76c2bbb1ea36" containerName="revoke-keys"
Oct 30 20:21:37.372174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.372055    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0f77e30-4745-4424-b7e3-76c2bbb1ea36" containerName="revoke-keys"
Oct 30 20:21:37.372174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.372108    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d9e50323-5c6e-469a-a2cc-61fb34920f58" containerName="revoke-keys"
Oct 30 20:21:37.372174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.372133    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9e50323-5c6e-469a-a2cc-61fb34920f58" containerName="revoke-keys"
Oct 30 20:21:37.372339 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.372292    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0f77e30-4745-4424-b7e3-76c2bbb1ea36" containerName="revoke-keys"
Oct 30 20:21:37.372339 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.372317    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9e50323-5c6e-469a-a2cc-61fb34920f58" containerName="revoke-keys"
Oct 30 20:21:37.375831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.375810    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.387425 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.387398    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:21:37.387539 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.387493    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:21:37.395566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.387839    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"]
Oct 30 20:21:37.415314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.415281    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42dh\" (UniqueName: \"kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.415455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.415430    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.415664 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.415639    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.516783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.516746    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.516951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.516857    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42dh\" (UniqueName: \"kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.516951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.516947    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.517610 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.517583    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.519232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.519207    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.530184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.530156    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42dh\" (UniqueName: \"kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh\") pod \"maas-api-revoke-sub-03eedbe0-l282n\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.555378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.555343    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:37.700783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.700702    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:37.983736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:37.983706    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"]
Oct 30 20:21:37.984258 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:37.984231    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8108047e_dc3f_4b7c_80a8_78ad3dcec743.slice/crio-b6c2cce7bac787e4fc5bf82bc8fe684ce84f92961e7c70c12dbd6048508521fb WatchSource:0}: Error finding container b6c2cce7bac787e4fc5bf82bc8fe684ce84f92961e7c70c12dbd6048508521fb: Status 404 returned error can't find the container with id b6c2cce7bac787e4fc5bf82bc8fe684ce84f92961e7c70c12dbd6048508521fb
Oct 30 20:21:38.319014 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:38.318924    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:38.370414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:38.370380    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n" event={"ID":"8108047e-dc3f-4b7c-80a8-78ad3dcec743","Type":"ContainerStarted","Data":"49763cdc5464c64c38a7a3faaeb5c1cbe01eea9e277b3c7a71bd2dc94fc85223"}
Oct 30 20:21:38.370601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:38.370431    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n" event={"ID":"8108047e-dc3f-4b7c-80a8-78ad3dcec743","Type":"ContainerStarted","Data":"b6c2cce7bac787e4fc5bf82bc8fe684ce84f92961e7c70c12dbd6048508521fb"}
Oct 30 20:21:38.393890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:38.393823    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n" podStartSLOduration=1.3938028820000001 podStartE2EDuration="1.393802882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:21:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:21:38.390375358 +0000 UTC m=+1018.589948309" watchObservedRunningTime="2026-10-30 20:21:38.393802882 +0000 UTC m=+1018.593375811"
Oct 30 20:21:39.278385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:39.278348    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:39.379534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:39.379503    6389 generic.go:356] "Generic (PLEG): container finished" podID="8108047e-dc3f-4b7c-80a8-78ad3dcec743" containerID="49763cdc5464c64c38a7a3faaeb5c1cbe01eea9e277b3c7a71bd2dc94fc85223" exitCode=0
Oct 30 20:21:39.379708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:39.379585    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n" event={"ID":"8108047e-dc3f-4b7c-80a8-78ad3dcec743","Type":"ContainerDied","Data":"49763cdc5464c64c38a7a3faaeb5c1cbe01eea9e277b3c7a71bd2dc94fc85223"}
Oct 30 20:21:40.052742 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.052707    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:40.549232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.549202    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:40.658476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.658395    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca\") pod \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") "
Oct 30 20:21:40.658653 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.658500    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca\") pod \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") "
Oct 30 20:21:40.658653 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.658646    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh\" (UniqueName: \"kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh\") pod \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\" (UID: \"8108047e-dc3f-4b7c-80a8-78ad3dcec743\") "
Oct 30 20:21:40.658913 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.658888    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca" pod "8108047e-dc3f-4b7c-80a8-78ad3dcec743" (UID: "8108047e-dc3f-4b7c-80a8-78ad3dcec743"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:40.659185 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.659165    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:40.660730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.660707    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca" pod "8108047e-dc3f-4b7c-80a8-78ad3dcec743" (UID: "8108047e-dc3f-4b7c-80a8-78ad3dcec743"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:40.661036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.661014    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh" pod "8108047e-dc3f-4b7c-80a8-78ad3dcec743" (UID: "8108047e-dc3f-4b7c-80a8-78ad3dcec743"). InnerVolumeSpecName "kube-api-access-t42dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:40.760487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.760451    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t42dh\" (UniqueName: \"kubernetes.io/projected/8108047e-dc3f-4b7c-80a8-78ad3dcec743-kube-api-access-t42dh\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:40.760487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:40.760489    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8108047e-dc3f-4b7c-80a8-78ad3dcec743-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:41.120168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.119529    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"]
Oct 30 20:21:41.121392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.121340    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:41.121550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.121429    6389 cpu_manager.go:414] "containerMap: removing container" podUID="8108047e-dc3f-4b7c-80a8-78ad3dcec743" containerName="revoke-keys"
Oct 30 20:21:41.121550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.121451    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8108047e-dc3f-4b7c-80a8-78ad3dcec743" containerName="revoke-keys"
Oct 30 20:21:41.122291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.121959    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="8108047e-dc3f-4b7c-80a8-78ad3dcec743" containerName="revoke-keys"
Oct 30 20:21:41.126290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.126266    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.131486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.131459    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5197ec\"/\"distinct-w1-5197ec-kserve-self-signed-certs\""
Oct 30 20:21:41.135651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.135629    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"]
Oct 30 20:21:41.165432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.165403    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.165622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.165453    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.165622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.165476    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.165622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.165592    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.165622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.165617    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvzll\" (UniqueName: \"kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.165861 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.165658    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.241105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.241068    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:41.273966 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.273926    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.274692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.274652    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvzll\" (UniqueName: \"kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.274847 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.274735    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.274971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.274945    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.275028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.275001    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.275082 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.275028    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.275184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.274589    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.275420 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.275399    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.276208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.276181    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.311157 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.280912    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.311157 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.303216    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvzll\" (UniqueName: \"kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.327139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.319424    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm\") pod \"distinct-w1-5197ec-kserve-58647b6cdb-q2f7b\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.397985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.397880    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n" event={"ID":"8108047e-dc3f-4b7c-80a8-78ad3dcec743","Type":"ContainerDied","Data":"b6c2cce7bac787e4fc5bf82bc8fe684ce84f92961e7c70c12dbd6048508521fb"}
Oct 30 20:21:41.397985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.397909    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"
Oct 30 20:21:41.397985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.397943    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6c2cce7bac787e4fc5bf82bc8fe684ce84f92961e7c70c12dbd6048508521fb"
Oct 30 20:21:41.447388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.447309    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:41.747776 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:41.747749    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42f2aa64_eccc_470b_8500_a0b0a2440c41.slice/crio-37f47ae67010b6b91ac47f82f3ed994fef9b9664a4c85fa95bad74ec0d45907b WatchSource:0}: Error finding container 37f47ae67010b6b91ac47f82f3ed994fef9b9664a4c85fa95bad74ec0d45907b: Status 404 returned error can't find the container with id 37f47ae67010b6b91ac47f82f3ed994fef9b9664a4c85fa95bad74ec0d45907b
Oct 30 20:21:41.748090 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.747865    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"]
Oct 30 20:21:41.781423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.781398    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"]
Oct 30 20:21:41.787051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.787032    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.791341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.791321    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:21:41.791460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.791382    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:21:41.795763 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.795741    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"]
Oct 30 20:21:41.883085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.883051    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.883265 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.883212    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv896\" (UniqueName: \"kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.883378 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.883364    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.984415 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.984381    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.984598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.984490    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv896\" (UniqueName: \"kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.984664 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.984650    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.985031 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.985010    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.987191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.987171    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:41.998462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:41.998400    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv896\" (UniqueName: \"kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896\") pod \"maas-api-revoke-sub-1244749a-552gz\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:42.026661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.026635    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:42.129814 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.129785    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:42.406710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.406624    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" event={"ID":"42f2aa64-eccc-470b-8500-a0b0a2440c41","Type":"ContainerStarted","Data":"ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab"}
Oct 30 20:21:42.406710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.406666    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" event={"ID":"42f2aa64-eccc-470b-8500-a0b0a2440c41","Type":"ContainerStarted","Data":"37f47ae67010b6b91ac47f82f3ed994fef9b9664a4c85fa95bad74ec0d45907b"}
Oct 30 20:21:42.419959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.419930    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"]
Oct 30 20:21:42.420616 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:42.420596    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca1a0c2_8981_4feb_b671_d7e86fc62070.slice/crio-cd79e497c7c47ea93c7456d7ac005d644095bad0ee07d214398cc1123d097e4c WatchSource:0}: Error finding container cd79e497c7c47ea93c7456d7ac005d644095bad0ee07d214398cc1123d097e4c: Status 404 returned error can't find the container with id cd79e497c7c47ea93c7456d7ac005d644095bad0ee07d214398cc1123d097e4c
Oct 30 20:21:42.432393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.432352    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podStartSLOduration=1.432340141 podStartE2EDuration="1.432340141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:21:42.430797358 +0000 UTC m=+1022.630370287" watchObservedRunningTime="2026-10-30 20:21:42.432340141 +0000 UTC m=+1022.631913068"
Oct 30 20:21:42.448356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:42.448326    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:43.174302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:43.174203    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:43.415006 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:43.414971    6389 generic.go:356] "Generic (PLEG): container finished" podID="3ca1a0c2-8981-4feb-b671-d7e86fc62070" containerID="b1ff398d2c3ed53b507cec23b94d96423ba6b391c992b5ef39eb47849be2be0a" exitCode=0
Oct 30 20:21:43.415198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:43.415046    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz" event={"ID":"3ca1a0c2-8981-4feb-b671-d7e86fc62070","Type":"ContainerDied","Data":"b1ff398d2c3ed53b507cec23b94d96423ba6b391c992b5ef39eb47849be2be0a"}
Oct 30 20:21:43.415198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:43.415095    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz" event={"ID":"3ca1a0c2-8981-4feb-b671-d7e86fc62070","Type":"ContainerStarted","Data":"cd79e497c7c47ea93c7456d7ac005d644095bad0ee07d214398cc1123d097e4c"}
Oct 30 20:21:43.951855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:43.951792    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:44.574379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.574355    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:44.722190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.722162    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca\") pod \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") "
Oct 30 20:21:44.722372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.722218    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca\") pod \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") "
Oct 30 20:21:44.722372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.722246    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896\" (UniqueName: \"kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896\") pod \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\" (UID: \"3ca1a0c2-8981-4feb-b671-d7e86fc62070\") "
Oct 30 20:21:44.722571 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.722547    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca" pod "3ca1a0c2-8981-4feb-b671-d7e86fc62070" (UID: "3ca1a0c2-8981-4feb-b671-d7e86fc62070"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:44.724303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.724277    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca" pod "3ca1a0c2-8981-4feb-b671-d7e86fc62070" (UID: "3ca1a0c2-8981-4feb-b671-d7e86fc62070"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:44.724431 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.724345    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896" pod "3ca1a0c2-8981-4feb-b671-d7e86fc62070" (UID: "3ca1a0c2-8981-4feb-b671-d7e86fc62070"). InnerVolumeSpecName "kube-api-access-pv896". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:44.823971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.823934    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:44.823971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.823963    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca1a0c2-8981-4feb-b671-d7e86fc62070-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:44.823971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.823974    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv896\" (UniqueName: \"kubernetes.io/projected/3ca1a0c2-8981-4feb-b671-d7e86fc62070-kube-api-access-pv896\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:44.863699 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:44.863666    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:45.430877 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:45.430849    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"
Oct 30 20:21:45.431078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:45.430887    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz" event={"ID":"3ca1a0c2-8981-4feb-b671-d7e86fc62070","Type":"ContainerDied","Data":"cd79e497c7c47ea93c7456d7ac005d644095bad0ee07d214398cc1123d097e4c"}
Oct 30 20:21:45.431078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:45.430943    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd79e497c7c47ea93c7456d7ac005d644095bad0ee07d214398cc1123d097e4c"
Oct 30 20:21:45.655604 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:45.655566    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:46.387899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:46.387870    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:47.202182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:47.202150    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:48.230079 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:48.230040    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:49.088095 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:49.088056    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:51.208262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:51.208210    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:51.450289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:51.450250    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:21:51.950249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:51.950202    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:52.192157 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.191240    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:52.229144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.225251    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:52.263631 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.263581    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:52.264288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.264256    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:21:52.267085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.267058    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:21:52.286224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.286186    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:52.286645 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.286226    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:52.449372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.449321    6389 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/health\": context deadline exceeded"
Oct 30 20:21:52.469762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.469720    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:21:52.470073 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:52.470042    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.134154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.132238    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.134154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.132858    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:21:53.207267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.207220    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.254579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.235254    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:21:53.292144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.263223    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:21:53.297142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.271247    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.415768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.295334    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:21:53.416514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.416486    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:21:53.416770 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.416754    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:21:53.417929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.295406    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.418393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.312558    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.419651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.419633    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:21:53.466946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.466909    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:21:53.467245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.467204    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.609138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.605179    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"]
Oct 30 20:21:53.609138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.606017    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3ca1a0c2-8981-4feb-b671-d7e86fc62070" containerName="revoke-keys"
Oct 30 20:21:53.609138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.606037    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ca1a0c2-8981-4feb-b671-d7e86fc62070" containerName="revoke-keys"
Oct 30 20:21:53.609138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.606294    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ca1a0c2-8981-4feb-b671-d7e86fc62070" containerName="revoke-keys"
Oct 30 20:21:53.655022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.654976    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.978277 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.978239    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:21:53.978729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.978697    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:53.979620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.978641    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:53.988550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.988186    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:21:53.988550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:53.988328    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:21:54.053844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.053813    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:21:54.054132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.054100    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"]
Oct 30 20:21:54.054676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.054657    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:21:54.081737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.081697    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.115315 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.115284    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.115773 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.115745    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8mz4\" (UniqueName: \"kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.144143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.132731    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:54.237196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.220702    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.266138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.237563    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.267091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.258445    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:21:54.267507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.267379    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:21:54.267796 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.261172    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.268543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.260439    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:54.268870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.268495    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:21:54.287136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.285787    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8mz4\" (UniqueName: \"kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.289199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.288336    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.399142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.393636    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:21:54.399142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.394695    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:21:54.460346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.460308    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 20:21:54.479139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.462240    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:21:54.583486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.583396    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8mz4\" (UniqueName: \"kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4\") pod \"maas-api-revoke-sub-dd41f1f2-9gt9m\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.643975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.643947    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:54.652023 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.644112    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:21:54.771044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.770905    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:21:54.771044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.770968    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:21:54.771255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:54.771133    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:55.076209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:55.076179    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"]
Oct 30 20:21:55.076462 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:21:55.076428    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e66127_3636_4dc8_b8ec_2749c9265124.slice/crio-0391e35400f3bb1a5f43c02d788e8caf1d114a0b2a2af58d2bf3d3a2119f701e WatchSource:0}: Error finding container 0391e35400f3bb1a5f43c02d788e8caf1d114a0b2a2af58d2bf3d3a2119f701e: Status 404 returned error can't find the container with id 0391e35400f3bb1a5f43c02d788e8caf1d114a0b2a2af58d2bf3d3a2119f701e
Oct 30 20:21:55.798822 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:21:55.798787    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:21:55.875448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:55.875389    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m" event={"ID":"21e66127-3636-4dc8-b8ec-2749c9265124","Type":"ContainerStarted","Data":"19ddb4f40f053f04d7fbe6da7defc2a881927a6ce81aac97128dc94bad937bc5"}
Oct 30 20:21:55.875700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:55.875461    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m" event={"ID":"21e66127-3636-4dc8-b8ec-2749c9265124","Type":"ContainerStarted","Data":"0391e35400f3bb1a5f43c02d788e8caf1d114a0b2a2af58d2bf3d3a2119f701e"}
Oct 30 20:21:55.904381 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:55.904328    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m" podStartSLOduration=2.904314282 podStartE2EDuration="2.904314282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:21:55.901445654 +0000 UTC m=+1036.101018586" watchObservedRunningTime="2026-10-30 20:21:55.904314282 +0000 UTC m=+1036.103887255"
Oct 30 20:21:56.207189 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:56.207078    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:56.885082 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:56.885045    6389 generic.go:356] "Generic (PLEG): container finished" podID="21e66127-3636-4dc8-b8ec-2749c9265124" containerID="19ddb4f40f053f04d7fbe6da7defc2a881927a6ce81aac97128dc94bad937bc5" exitCode=0
Oct 30 20:21:56.885479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:56.885159    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m" event={"ID":"21e66127-3636-4dc8-b8ec-2749c9265124","Type":"ContainerDied","Data":"19ddb4f40f053f04d7fbe6da7defc2a881927a6ce81aac97128dc94bad937bc5"}
Oct 30 20:21:57.581953 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:57.581898    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:58.044417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.044394    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:58.215087 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.214997    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca\") pod \"21e66127-3636-4dc8-b8ec-2749c9265124\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") "
Oct 30 20:21:58.215087 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.215052    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca\") pod \"21e66127-3636-4dc8-b8ec-2749c9265124\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") "
Oct 30 20:21:58.215324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.215283    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4\" (UniqueName: \"kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4\") pod \"21e66127-3636-4dc8-b8ec-2749c9265124\" (UID: \"21e66127-3636-4dc8-b8ec-2749c9265124\") "
Oct 30 20:21:58.215523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.215439    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca" pod "21e66127-3636-4dc8-b8ec-2749c9265124" (UID: "21e66127-3636-4dc8-b8ec-2749c9265124"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:58.215955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.215913    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:58.217439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.217410    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca" pod "21e66127-3636-4dc8-b8ec-2749c9265124" (UID: "21e66127-3636-4dc8-b8ec-2749c9265124"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:58.217596 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.217575    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4" pod "21e66127-3636-4dc8-b8ec-2749c9265124" (UID: "21e66127-3636-4dc8-b8ec-2749c9265124"). InnerVolumeSpecName "kube-api-access-f8mz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:58.317057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.317020    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8mz4\" (UniqueName: \"kubernetes.io/projected/21e66127-3636-4dc8-b8ec-2749c9265124-kube-api-access-f8mz4\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:58.317057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.317056    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21e66127-3636-4dc8-b8ec-2749c9265124-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:58.495684 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.495601    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:58.901026 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.900949    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"
Oct 30 20:21:58.901182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.900944    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m" event={"ID":"21e66127-3636-4dc8-b8ec-2749c9265124","Type":"ContainerDied","Data":"0391e35400f3bb1a5f43c02d788e8caf1d114a0b2a2af58d2bf3d3a2119f701e"}
Oct 30 20:21:58.901182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:58.901100    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0391e35400f3bb1a5f43c02d788e8caf1d114a0b2a2af58d2bf3d3a2119f701e"
Oct 30 20:21:59.260505 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:59.260469    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:59.506390 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:59.506352    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:21:59.646980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:59.646891    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"]
Oct 30 20:21:59.730982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:21:59.730951    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:00.735893 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:00.735758    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:01.457272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.457241    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"]
Oct 30 20:22:01.457850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.457836    6389 cpu_manager.go:414] "containerMap: removing container" podUID="21e66127-3636-4dc8-b8ec-2749c9265124" containerName="revoke-keys"
Oct 30 20:22:01.457925 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.457852    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21e66127-3636-4dc8-b8ec-2749c9265124" containerName="revoke-keys"
Oct 30 20:22:01.457972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.457963    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="21e66127-3636-4dc8-b8ec-2749c9265124" containerName="revoke-keys"
Oct 30 20:22:01.461462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.461442    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:22:01.461803 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.461777    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.467002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.466981    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:22:01.467421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.467400    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:22:01.471514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.471492    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"]
Oct 30 20:22:01.476474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.476452    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:22:01.659658 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.659628    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.659828 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.659665    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.659929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.659909    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vj5\" (UniqueName: \"kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.761588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.761552    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49vj5\" (UniqueName: \"kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.762022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.761639    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.762022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.761674    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.762440 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.762411    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.764077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.764058    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.776105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.776081    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vj5\" (UniqueName: \"kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5\") pod \"maas-api-revoke-sub-93041bbd-gmw5b\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.778203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.778185    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:01.808718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:01.808693    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:02.069420 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:02.068310    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"]
Oct 30 20:22:02.652972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:02.652877    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:02.934297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:02.934265    6389 generic.go:356] "Generic (PLEG): container finished" podID="f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" containerID="9be3ab59e3a18b98aa8088067b795c0be5f5ff718906f563d18fdd8f0d347e56" exitCode=0
Oct 30 20:22:02.934682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:02.934347    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b" event={"ID":"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117","Type":"ContainerDied","Data":"9be3ab59e3a18b98aa8088067b795c0be5f5ff718906f563d18fdd8f0d347e56"}
Oct 30 20:22:02.934682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:02.934394    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b" event={"ID":"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117","Type":"ContainerStarted","Data":"848a959c6319c3d5f912132c994240ae167ae48b11a6b5155568daffbdde9ed5"}
Oct 30 20:22:03.260829 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:03.260800    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:04.097981 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.097959    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:04.192756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.192671    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca\") pod \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") "
Oct 30 20:22:04.192756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.192702    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5\" (UniqueName: \"kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5\") pod \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") "
Oct 30 20:22:04.192978 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.192796    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca\") pod \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\" (UID: \"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117\") "
Oct 30 20:22:04.193071 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.193048    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca" pod "f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" (UID: "f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:04.193285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.193266    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:04.194708 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.194676    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5" pod "f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" (UID: "f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117"). InnerVolumeSpecName "kube-api-access-49vj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:04.194872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.194854    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca" pod "f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" (UID: "f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:04.277017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.276992    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:04.294733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.294707    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49vj5\" (UniqueName: \"kubernetes.io/projected/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-kube-api-access-49vj5\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:04.294733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.294732    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:04.465175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.465148    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"]
Oct 30 20:22:04.482682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.482654    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"]
Oct 30 20:22:04.586851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.586818    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:04.949436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.949368    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"
Oct 30 20:22:04.949566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.949373    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b" event={"ID":"f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117","Type":"ContainerDied","Data":"848a959c6319c3d5f912132c994240ae167ae48b11a6b5155568daffbdde9ed5"}
Oct 30 20:22:04.949566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:04.949476    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="848a959c6319c3d5f912132c994240ae167ae48b11a6b5155568daffbdde9ed5"
Oct 30 20:22:05.125927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:05.125891    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:05.280715 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:05.280677    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:05.798029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:05.797993    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"]
Oct 30 20:22:05.851695 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:05.851659    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:05.956344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:05.956315    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:06.537656 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:06.537621    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:07.567058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.567020    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:07.963205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.963102    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"]
Oct 30 20:22:07.964067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.964041    6389 cpu_manager.go:414] "containerMap: removing container" podUID="f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" containerName="revoke-keys"
Oct 30 20:22:07.964233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.964216    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" containerName="revoke-keys"
Oct 30 20:22:07.964415 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.964394    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" containerName="revoke-keys"
Oct 30 20:22:07.968670 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.968641    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:07.979711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.979635    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5197ec\"/\"distinct-2-w1-5197ec-kserve-self-signed-certs\""
Oct 30 20:22:07.981474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:07.981447    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"]
Oct 30 20:22:08.014267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.014237    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:08.049834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.049798    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.050001 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.049964    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.050184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.050157    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.050226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.050202    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq6q5\" (UniqueName: \"kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.050342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.050325    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.050430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.050415    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.124416 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.124380    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:08.151527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.151488    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.151777 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.151758    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq6q5\" (UniqueName: \"kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.151975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.151956    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.152178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.152158    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.152355 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.152337    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.152562 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.152544    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.153391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.153364    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.156211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.156183    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.156742 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.156709    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.156820 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.156801    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.159246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.159220    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.170385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.170356    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq6q5\" (UniqueName: \"kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5\") pod \"distinct-2-w1-5197ec-kserve-77f8785557-x96nm\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") " pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.209366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.209325    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"]
Oct 30 20:22:08.213719 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.213676    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.218888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.218859    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:22:08.219199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.219177    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:22:08.227703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.227679    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"]
Oct 30 20:22:08.271851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.271816    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.272400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.272285    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.273025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.272640    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk67w\" (UniqueName: \"kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.291391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.291349    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:08.292511 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.292257    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:08.377128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.374395    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.377128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.374464    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.377128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.374691    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk67w\" (UniqueName: \"kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.377128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.376416    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.378264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.378167    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.388985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.388759    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk67w\" (UniqueName: \"kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w\") pod \"maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.575048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.574998    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:08.579651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.579625    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"]
Oct 30 20:22:08.579746 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:08.579626    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod452532ae_6a10_49ca_85dd_c9480c1a0cc8.slice/crio-e32b129b8634059cdf69455e95eb4f2dd3ef9cae0db7777afa3f9da6618b8631 WatchSource:0}: Error finding container e32b129b8634059cdf69455e95eb4f2dd3ef9cae0db7777afa3f9da6618b8631: Status 404 returned error can't find the container with id e32b129b8634059cdf69455e95eb4f2dd3ef9cae0db7777afa3f9da6618b8631
Oct 30 20:22:08.863223 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:08.863196    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c977a9_c12f_4686_9249_52dc228c2cb5.slice/crio-b497c9328a824317558f74de9397e84ee87fb8747a93b5364955d5e6f09e99f0 WatchSource:0}: Error finding container b497c9328a824317558f74de9397e84ee87fb8747a93b5364955d5e6f09e99f0: Status 404 returned error can't find the container with id b497c9328a824317558f74de9397e84ee87fb8747a93b5364955d5e6f09e99f0
Oct 30 20:22:08.863342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.863275    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"]
Oct 30 20:22:08.867623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.867224    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:08.983283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.983245    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd" event={"ID":"31c977a9-c12f-4686-9249-52dc228c2cb5","Type":"ContainerStarted","Data":"b497c9328a824317558f74de9397e84ee87fb8747a93b5364955d5e6f09e99f0"}
Oct 30 20:22:08.985029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.985001    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" event={"ID":"452532ae-6a10-49ca-85dd-c9480c1a0cc8","Type":"ContainerStarted","Data":"e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002"}
Oct 30 20:22:08.985186 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:08.985037    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" event={"ID":"452532ae-6a10-49ca-85dd-c9480c1a0cc8","Type":"ContainerStarted","Data":"e32b129b8634059cdf69455e95eb4f2dd3ef9cae0db7777afa3f9da6618b8631"}
Oct 30 20:22:09.007414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.007352    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" podStartSLOduration=2.007338457 podStartE2EDuration="2.007338457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:09.006535263 +0000 UTC m=+1049.206108192" watchObservedRunningTime="2026-10-30 20:22:09.007338457 +0000 UTC m=+1049.206911494"
Oct 30 20:22:09.293379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.293345    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:09.579083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.579004    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"]
Oct 30 20:22:09.583398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.583376    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.599253 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.599221    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"]
Oct 30 20:22:09.603935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.603910    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"]
Oct 30 20:22:09.608998 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.608971    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.621349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.620880    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"]
Oct 30 20:22:09.693438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693404    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.693607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693481    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.693607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693556    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cnd6\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.693607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693592    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.693757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693613    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.693757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693637    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.693867 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693846    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.693902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693881    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.693948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693932    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.694012 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.693997    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.694048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694040    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694060    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694173 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694137    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694231 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694208    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694268 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694234    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpkn\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694268    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694358    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.694409 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.694388    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.795911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.795875    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.795930    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.795994    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796069    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796088    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796107    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796177    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796218    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796233    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpkn\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796263    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796296    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796312    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796338    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796371    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796402    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cnd6\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796403    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796427    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796454    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796484    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796580    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.796850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.796831    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.797236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.797047    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.798081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.797332    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.798081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.797457    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.798081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.797564    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.798081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.797650    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.798081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.798019    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.798487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.798236    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.799797 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.799775    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.799797 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.799790    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.799996 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.799972    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.801049 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.801024    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.808524 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.808497    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.809747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.809719    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpkn\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.810474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.810454    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.810858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.810834    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cnd6\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.851157 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.851073    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"]
Oct 30 20:22:09.881558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.881526    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"]
Oct 30 20:22:09.888830 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.888800    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:09.904212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.904183    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"]
Oct 30 20:22:09.907963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.907739    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:09.929220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.928347    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:09.932281 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:09.932073    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:10.002659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.002628    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.002854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.002787    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.002854 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.002825    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.002991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.002882    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.002991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.002938    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.002991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.002976    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.003151 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.003002    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5qt8\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.003151 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.003089    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.003244 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.003231    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.008589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.008567    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"]
Oct 30 20:22:10.029209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.027485    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"]
Oct 30 20:22:10.029209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.027574    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.038657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.035212    6389 generic.go:356] "Generic (PLEG): container finished" podID="31c977a9-c12f-4686-9249-52dc228c2cb5" containerID="2e7a9746f09229ed9fd906f8ca4ef25e8ff314745e4f890289e361c6225b24bf" exitCode=0
Oct 30 20:22:10.038657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.035702    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd" event={"ID":"31c977a9-c12f-4686-9249-52dc228c2cb5","Type":"ContainerDied","Data":"2e7a9746f09229ed9fd906f8ca4ef25e8ff314745e4f890289e361c6225b24bf"}
Oct 30 20:22:10.057889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.057791    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:10.104942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.104838    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.104942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.104882    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.104942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.104922    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r64xv\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.105204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.104974    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.105339 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.105315    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.105917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.105455    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.105917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.105872    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.105917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.105901    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.105943    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.106128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.105986    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106037    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.106128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106065    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.106128 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106095    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106150    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106176    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qt8\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106455    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106624 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106555    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.106692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106621    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.106785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106769    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.106882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106842    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.107007 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.106987    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.107192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.107110    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.107631 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.107600    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.109844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.109822    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.109942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.109926    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.120510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.120477    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.120644 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.120563    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5qt8\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8\") pod \"e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") " pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.162456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.162386    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:10.208283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208241    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208300    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r64xv\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208596    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208672 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208661    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208704    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208805 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208749    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208805 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208753    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208822    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.208971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208928    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.209027 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208968    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.209027 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.208973    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.209263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.209247    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.209734 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.209712    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.209989 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.209968    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.211920 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.211895    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.212216 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.212198    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.228848 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.228425    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.256450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.255628    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"]
Oct 30 20:22:10.256450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.255949    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:10.305840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.305029    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r64xv\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv\") pod \"e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") " pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.318677 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.318458    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.318677 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.318586    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.318869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.318691    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.321103 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:10.321075    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb503944_3ca8_4861_a51d_b1e2448ef651.slice/crio-83f3de7200a62b5682e9f1476d9e009a7fa8c1963410452c075321ad2209c931 WatchSource:0}: Error finding container 83f3de7200a62b5682e9f1476d9e009a7fa8c1963410452c075321ad2209c931: Status 404 returned error can't find the container with id 83f3de7200a62b5682e9f1476d9e009a7fa8c1963410452c075321ad2209c931
Oct 30 20:22:10.330002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.329974    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"]
Oct 30 20:22:10.335280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.335236    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.335389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.335358    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.335460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.335406    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.336212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.336164    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"]
Oct 30 20:22:10.358187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.357519    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:10.456582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.456438    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:10.649110 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.649075    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"]
Oct 30 20:22:10.652654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.652620    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.652774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.652706    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.652774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.652750    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.736913 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.736652    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"]
Oct 30 20:22:10.739359 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.739319    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.739469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.739413    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.739469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.739452    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:10.799159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:10.798928    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerName="payload-pre-processing" containerID="cri-o://14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381" gracePeriod=25
Oct 30 20:22:11.116528 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.116491    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" event={"ID":"cb503944-3ca8-4861-a51d-b1e2448ef651","Type":"ContainerStarted","Data":"ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e"}
Oct 30 20:22:11.116782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.116767    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" event={"ID":"cb503944-3ca8-4861-a51d-b1e2448ef651","Type":"ContainerStarted","Data":"83f3de7200a62b5682e9f1476d9e009a7fa8c1963410452c075321ad2209c931"}
Oct 30 20:22:11.120086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.120059    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" event={"ID":"d7bfa1e7-769b-4f38-bc5c-677f603542c0","Type":"ContainerStarted","Data":"5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1"}
Oct 30 20:22:11.151174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.150581    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" event={"ID":"d7bfa1e7-769b-4f38-bc5c-677f603542c0","Type":"ContainerStarted","Data":"a13d8eb2a5e772dad4bdeedc915c27197055004551fd3e4e99c81af3b9736da0"}
Oct 30 20:22:11.199510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.199401    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" event={"ID":"e430a09e-18f3-471a-a53b-a0fcf7b8e626","Type":"ContainerStarted","Data":"9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb"}
Oct 30 20:22:11.199510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.199465    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" event={"ID":"e430a09e-18f3-471a-a53b-a0fcf7b8e626","Type":"ContainerStarted","Data":"8ea7d0acef3ca9986c77539c6e03150401f5f2a3973272195e2b88ddcd830dfe"}
Oct 30 20:22:11.217487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.217415    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podStartSLOduration=2.217392393 podStartE2EDuration="2.217392393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:11.195620911 +0000 UTC m=+1051.395193839" watchObservedRunningTime="2026-10-30 20:22:11.217392393 +0000 UTC m=+1051.416965323"
Oct 30 20:22:11.312144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.286765    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" event={"ID":"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c","Type":"ContainerStarted","Data":"50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b"}
Oct 30 20:22:11.312144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.286831    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" event={"ID":"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c","Type":"ContainerStarted","Data":"bf4bcfb36d4addd582688b5f56e707d68b1c43315c1bb993cbfa47b154f2cf85"}
Oct 30 20:22:11.390173 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.390018    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:11.423452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.423420    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:11.423740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.423702    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:11.451149 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.425677    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podStartSLOduration=2.425657502 podStartE2EDuration="2.425657502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:11.387620696 +0000 UTC m=+1051.587193631" watchObservedRunningTime="2026-10-30 20:22:11.425657502 +0000 UTC m=+1051.625230432"
Oct 30 20:22:11.495140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.490827    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podStartSLOduration=2.490805495 podStartE2EDuration="2.490805495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:11.307524136 +0000 UTC m=+1051.507097065" watchObservedRunningTime="2026-10-30 20:22:11.490805495 +0000 UTC m=+1051.690378426"
Oct 30 20:22:11.552137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.551519    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podStartSLOduration=2.551495607 podStartE2EDuration="2.551495607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:11.415028523 +0000 UTC m=+1051.614601452" watchObservedRunningTime="2026-10-30 20:22:11.551495607 +0000 UTC m=+1051.751068540"
Oct 30 20:22:11.700344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.648775    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:11.910364 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.910317    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:11.910651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.910620    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:11.931071 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.931040    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:11.931401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:11.931350    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:12.054289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.054257    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:12.054586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.054568    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:12.273108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.273080    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:12.371312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.371219    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:12.376399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.376265    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:12.396599 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.396544    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:12.396965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.274651    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:12.499211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.499163    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/health\": context deadline exceeded"
Oct 30 20:22:12.499662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.499627    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:12.509633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.509602    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:12.669421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.653784    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:12.669421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.653650    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:12.669421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.653606    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:12.787691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.787631    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:12.788710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.788664    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 20:22:12.873709 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.872096    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:12.912608 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.912570    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:12.912874 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.912843    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:12.925185 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.922337    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:12.929566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.929536    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:12.929791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.929765    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:12.953851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.953806    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w\" (UniqueName: \"kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w\") pod \"31c977a9-c12f-4686-9249-52dc228c2cb5\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") "
Oct 30 20:22:12.954698 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.954656    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca\") pod \"31c977a9-c12f-4686-9249-52dc228c2cb5\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") "
Oct 30 20:22:12.955408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.955390    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca\") pod \"31c977a9-c12f-4686-9249-52dc228c2cb5\" (UID: \"31c977a9-c12f-4686-9249-52dc228c2cb5\") "
Oct 30 20:22:12.956058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.955875    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca" pod "31c977a9-c12f-4686-9249-52dc228c2cb5" (UID: "31c977a9-c12f-4686-9249-52dc228c2cb5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:12.956824 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.956805    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.974511 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.974475    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w" pod "31c977a9-c12f-4686-9249-52dc228c2cb5" (UID: "31c977a9-c12f-4686-9249-52dc228c2cb5"). InnerVolumeSpecName "kube-api-access-mk67w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:12.985329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:12.983353    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca" pod "31c977a9-c12f-4686-9249-52dc228c2cb5" (UID: "31c977a9-c12f-4686-9249-52dc228c2cb5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:13.041169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.017546    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"]
Oct 30 20:22:13.060199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.024209    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.060199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.060103    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk67w\" (UniqueName: \"kubernetes.io/projected/31c977a9-c12f-4686-9249-52dc228c2cb5-kube-api-access-mk67w\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:13.060199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.060152    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31c977a9-c12f-4686-9249-52dc228c2cb5-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:13.065683 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.065286    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.167519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.167485    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:13.241302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.241256    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:13.268476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.268426    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.293609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.269589    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:13.294658 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.294627    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:13.362007 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.361535    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:13.362007 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.361603    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:13.412143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.407655    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.412143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.408377    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:22:13.427144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.416450    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"
Oct 30 20:22:13.456144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.429249    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd" event={"ID":"31c977a9-c12f-4686-9249-52dc228c2cb5","Type":"ContainerDied","Data":"b497c9328a824317558f74de9397e84ee87fb8747a93b5364955d5e6f09e99f0"}
Oct 30 20:22:13.456144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.429307    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b497c9328a824317558f74de9397e84ee87fb8747a93b5364955d5e6f09e99f0"
Oct 30 20:22:13.679183 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.662931    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.751276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.719438    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:13.751276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.731629    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:22:13.751276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.731888    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:13.782934 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.719716    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.866344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.864232    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:13.878198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.877722    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:22:13.889790 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.889750    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:22:13.889974 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.889857    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:13.890037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:13.890009    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:22:14.028853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.027772    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:14.028853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.027844    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:14.028853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.028559    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:14.101158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.059221    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:22:14.153204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.147756    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:22:14.203169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.193704    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:14.203169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.193781    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:14.207806 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.207775    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:22:14.244262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.221380    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:14.244530 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.244404    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:22:14.520414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.517948    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:22:14.622160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.604342    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:14.622160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.604400    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:14.622160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.614910    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:14.622160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.614961    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:14.742356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.742310    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" containerID="cri-o://5ad75c7182259906f37d5650003db3e6a01e45fb333da2f031b8117dc9e5ec52" gracePeriod=45
Oct 30 20:22:14.754142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.750774    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:14.762183 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.755217    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:14.870199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.853870    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing" containerID="cri-o://2d65b71b1fe1906001302a9322a06a1012fbe0778abf3f23b7d92caeaa3a2ef4" gracePeriod=25
Oct 30 20:22:14.954319 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.954282    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.959058    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.959090    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.959111    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.058448    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.347104    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.347192    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.195754    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:14.958193    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:15.350261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.348003    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:15.351195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.292677    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:15.378153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.378085    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:15.378514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.378485    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:15.389294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.388317    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:15.389294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.388373    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:15.630198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.626388    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:15.647198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.626861    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:15.647925 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.647874    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:15.648465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.648431    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:22:15.758886 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.758843    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:22:15.770055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.770013    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:15.813243 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.813180    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:15.813628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.813600    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 20:22:15.813732 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:15.813714    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:22:16.170539 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.170505    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:16.171613 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.171577    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:16.214345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.214251    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:16.214579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.214552    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:16.285422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.276409    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:16.285422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.276469    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:16.348770 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.348732    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:16.350054 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.350019    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:16.351949 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.351923    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:16.387762 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.387721    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:16.494093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.393967    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:16.494386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.494355    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:16.495744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.405282    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:16.509481 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.396834    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:22:16.651165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.637430    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:22:16.663155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.654866    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:16.663155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.656020    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 20:22:16.663155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.656609    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:16.663155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:16.657686    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:22:17.053152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.050584    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.088686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.088612    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.117975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.107934    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 20:22:17.117975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.108056    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:17.117975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.108081    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:17.323196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.320482    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:17.323196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.320589    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:17.405142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.383848    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.426147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.414455    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:17.426147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.414517    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:17.426147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.418055    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:17.571143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.547524    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:17.571143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.547588    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:17.571419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.571372    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:17.576070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.575373    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.653488 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.637835    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.725037 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.654664    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:17.725478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.668576    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:17.725796 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.668625    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.726311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.668659    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 20:22:17.908238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.907299    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:17.912028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.911903    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:17.912028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.911966    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:17.984143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:17.975526    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 20:22:18.007241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.007144    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:18.007443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.007257    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:18.067145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.051214    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.100353 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.100307    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.268558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.268520    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:18.280284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.280231    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:18.388168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.327199    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.349872    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.379292    6389 patch_prober.go:29] interesting pod/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.512990    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.502747    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.467580    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.380637    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:18.524200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.513660    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:18.572718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.572674    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:22:18.594147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.583176    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.726374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.726332    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.726986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.726955    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.727447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.727425    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.729236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.729211    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:18.775858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:18.775752    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 20:22:19.021402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.021365    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:19.021724 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.021680    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:19.022450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.022427    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:19.022626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.022604    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:19.058177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.058055    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.107415 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.107355    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.122141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.118486    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.268626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.268588    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:19.268895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.268865    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:19.324248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.323937    6389 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": context deadline exceeded"
Oct 30 20:22:19.360820 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.360784    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:19.361150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.361090    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:19.505667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.505542    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" containerID="cri-o://296c7e28e24a941ad5e2e6f39dc419c2b965e848bf195bf43351093486bacc2a" gracePeriod=45
Oct 30 20:22:19.515070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.515020    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.515415 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.515288    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.523264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.523241    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:22:19.597893 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.597787    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.617490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.617451    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": read tcp 10.134.0.2:35456->10.134.0.60:8000: read: connection reset by peer"
Oct 30 20:22:19.618286 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.618257    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Oct 30 20:22:19.618580 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.618556    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:19.653206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.653176    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:19.759406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.748398    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:22:19.759406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.748816    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:22:19.759406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.749107    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:19.802251 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.802194    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 20:22:19.918332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.908526    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:19.918332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.908585    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:19.949165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.929810    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:19.949593 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:19.949554    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:20.068018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.067980    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:20.073540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.072500    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.145286 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.117234    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.204450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.202005    6389 generic.go:356] "Generic (PLEG): container finished" podID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerID="5ad75c7182259906f37d5650003db3e6a01e45fb333da2f031b8117dc9e5ec52" exitCode=0
Oct 30 20:22:20.357194 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.335611    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:20.410888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.402232    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:20.410888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.388070    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" event={"ID":"1029303a-e800-4bf0-be8c-bbe05fcec90d","Type":"ContainerDied","Data":"5ad75c7182259906f37d5650003db3e6a01e45fb333da2f031b8117dc9e5ec52"}
Oct 30 20:22:20.410888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.346226    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Oct 30 20:22:20.410888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.362918    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:20.410888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.402375    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:20.410888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.410682    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.535158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.518278    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.606162 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.605329    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.622793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.622730    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.760303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.760261    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.761777 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.760534    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.761970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.760571    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:22:20.783887 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.783790    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 20:22:20.801344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.801305    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.898414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.854930    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh" podUID="df0c953c-ae59-46fa-8c05-4362c046e8db" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.20:6080/livez\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:20.898414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.876258    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:20.922460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.912635    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:20.922460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.912683    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:20.950309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.950264    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:20.950511 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.950328    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:21.001527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:20.982208    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-nxfsh" podUID="df0c953c-ae59-46fa-8c05-4362c046e8db" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.20:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.033225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.020961    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"]
Oct 30 20:22:21.169666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.169621    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:21.291188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.289778    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Oct 30 20:22:21.314143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.306353    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:21.314143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.306411    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:21.314143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.312426    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:21.344476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.344420    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.455066 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.455013    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:21.455408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.455353    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:21.594096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.572647    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.617801 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.617758    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.640458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.633479    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.811186 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.800391    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.859307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.807219    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.898767 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.807731    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.899873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.828422    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:22:21.900533 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.887438    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 20:22:21.901277 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.897284    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:21.902546 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.902530    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:21.915295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.915177    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:21.915295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.915229    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:22.002314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.970260    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:22.002314 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:21.970324    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:22.162901 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.162798    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:22.222341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.222307    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm\") pod \"1029303a-e800-4bf0-be8c-bbe05fcec90d\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") "
Oct 30 20:22:22.222991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.222595    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn\" (UniqueName: \"kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn\") pod \"1029303a-e800-4bf0-be8c-bbe05fcec90d\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") "
Oct 30 20:22:22.235330 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.235291    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir\") pod \"1029303a-e800-4bf0-be8c-bbe05fcec90d\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") "
Oct 30 20:22:22.280794 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.280747    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs\" (UniqueName: \"kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs\") pod \"1029303a-e800-4bf0-be8c-bbe05fcec90d\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") "
Oct 30 20:22:22.281258 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.281236    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home\") pod \"1029303a-e800-4bf0-be8c-bbe05fcec90d\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") "
Oct 30 20:22:22.281433 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.281407    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache\") pod \"1029303a-e800-4bf0-be8c-bbe05fcec90d\" (UID: \"1029303a-e800-4bf0-be8c-bbe05fcec90d\") "
Oct 30 20:22:22.283897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.283861    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir" pod "1029303a-e800-4bf0-be8c-bbe05fcec90d" (UID: "1029303a-e800-4bf0-be8c-bbe05fcec90d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:22.284819 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.284777    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.290235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.290204    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:22.290362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.290256    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:22.290362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.290336    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Oct 30 20:22:22.356919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.356879    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home" pod "1029303a-e800-4bf0-be8c-bbe05fcec90d" (UID: "1029303a-e800-4bf0-be8c-bbe05fcec90d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:22.384457 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.384413    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:22.412538 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.395801    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache" pod "1029303a-e800-4bf0-be8c-bbe05fcec90d" (UID: "1029303a-e800-4bf0-be8c-bbe05fcec90d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:22.412774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.397935    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:22.412938 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.412881    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:22.413438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.399299    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:22.413592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.413579    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:22.413875 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.413851    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn" pod "1029303a-e800-4bf0-be8c-bbe05fcec90d" (UID: "1029303a-e800-4bf0-be8c-bbe05fcec90d"). InnerVolumeSpecName "kube-api-access-cgdmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:22.444478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.444441    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm" pod "1029303a-e800-4bf0-be8c-bbe05fcec90d" (UID: "1029303a-e800-4bf0-be8c-bbe05fcec90d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:22.450280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.450250    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs" pod "1029303a-e800-4bf0-be8c-bbe05fcec90d" (UID: "1029303a-e800-4bf0-be8c-bbe05fcec90d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:22.469152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.466207    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.527372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.518360    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:22.527622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.527596    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgdmn\" (UniqueName: \"kubernetes.io/projected/1029303a-e800-4bf0-be8c-bbe05fcec90d-kube-api-access-cgdmn\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:22.539141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.537169    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1029303a-e800-4bf0-be8c-bbe05fcec90d-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:22.539141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.537205    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1029303a-e800-4bf0-be8c-bbe05fcec90d-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:22.573266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.573207    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.651261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.651213    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 20:22:22.653423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.651830    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.860144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.857225    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.872650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.872611    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t" event={"ID":"1029303a-e800-4bf0-be8c-bbe05fcec90d","Type":"ContainerDied","Data":"5403536f789c6dcd23ca89d82b93f244b819074f341d4d9ad254e2ff17f3292b"}
Oct 30 20:22:22.872919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.872898    6389 scope.go:122] "RemoveContainer" containerID="5ad75c7182259906f37d5650003db3e6a01e45fb333da2f031b8117dc9e5ec52"
Oct 30 20:22:22.873236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.873215    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"
Oct 30 20:22:22.905717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.905680    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.906222 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.906197    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:22.918459 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.913844    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:22:22.919392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.913892    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:22.919504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.919390    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:22.934140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.917604    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:22.934140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.933504    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:22.948140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.935000    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:22.948140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:22.935039    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:23.183717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.183623    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"]
Oct 30 20:22:23.190897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.190863    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-c646da/simulator-w0-c646da-kserve-557544f9b7-rq55t"]
Oct 30 20:22:23.290138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.285872    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Oct 30 20:22:23.290138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.286327    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:23.290138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.286372    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:23.364167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.360463    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:23.372240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.372197    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:23.427374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.427324    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:23.575144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.570093    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:22:23.577975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.576183    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:23.609315 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.609285    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:22:23.610748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.610561    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" containerID="cri-o://cfc26422c275c243022694e9fdbd6b0ab70c823e690e1a3680969c1c94577951" gracePeriod=41
Oct 30 20:22:23.654212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.654174    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:22:23.751328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.748361    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": EOF"
Oct 30 20:22:23.809810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.807890    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:22:23.879858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.877999    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": dial tcp 10.134.0.71:8000: connect: connection refused"
Oct 30 20:22:23.879858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.878155    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:22:23.922036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.918890    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:23.938495 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.938447    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:23.963136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.947580    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:23.963136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.947968    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 20:22:23.963136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.948332    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:22:23.963136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.948371    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 20:22:23.976990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.976491    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 20:22:23.976990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:23.976560    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 20:22:24.030343 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.030262    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" path="/var/lib/kubelet/pods/1029303a-e800-4bf0-be8c-bbe05fcec90d/volumes"
Oct 30 20:22:24.031105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.031075    6389 generic.go:356] "Generic (PLEG): container finished" podID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerID="296c7e28e24a941ad5e2e6f39dc419c2b965e848bf195bf43351093486bacc2a" exitCode=0
Oct 30 20:22:24.031933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.031913    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:24.032630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.032607    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" event={"ID":"7ab09e66-6512-431a-b3ab-0358b57baeac","Type":"ContainerDied","Data":"296c7e28e24a941ad5e2e6f39dc419c2b965e848bf195bf43351093486bacc2a"}
Oct 30 20:22:24.034266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.034235    6389 generic.go:356] "Generic (PLEG): container finished" podID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerID="cfc26422c275c243022694e9fdbd6b0ab70c823e690e1a3680969c1c94577951" exitCode=0
Oct 30 20:22:24.036536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.035351    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" event={"ID":"d1295ab6-c34b-4fb8-acc5-89cc095986fc","Type":"ContainerDied","Data":"cfc26422c275c243022694e9fdbd6b0ab70c823e690e1a3680969c1c94577951"}
Oct 30 20:22:24.091172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.089143    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:22:24.143211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.143184    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:22:24.181138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.180896    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:24.293491 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.293410    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:24.294059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.294026    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:22:24.296779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.296630    6389 patch_prober.go:29] interesting pod/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:22:24.299568 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.299539    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 20:22:24.300063 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.297378    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:22:24.358566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.358530    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 20:22:24.369578 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.369533    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 20:22:24.594890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.594781    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:22:24.637175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.637144    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm\") pod \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") "
Oct 30 20:22:24.638895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.638873    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir\") pod \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") "
Oct 30 20:22:24.639064 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.639049    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache\") pod \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") "
Oct 30 20:22:24.639239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.639223    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs\") pod \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") "
Oct 30 20:22:24.639415 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.639398    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh\" (UniqueName: \"kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh\") pod \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") "
Oct 30 20:22:24.639626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.639612    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home\") pod \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\" (UID: \"d1295ab6-c34b-4fb8-acc5-89cc095986fc\") "
Oct 30 20:22:24.640490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.640469    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home" pod "d1295ab6-c34b-4fb8-acc5-89cc095986fc" (UID: "d1295ab6-c34b-4fb8-acc5-89cc095986fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.641730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.641695    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir" pod "d1295ab6-c34b-4fb8-acc5-89cc095986fc" (UID: "d1295ab6-c34b-4fb8-acc5-89cc095986fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.642168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.642147    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache" pod "d1295ab6-c34b-4fb8-acc5-89cc095986fc" (UID: "d1295ab6-c34b-4fb8-acc5-89cc095986fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.647017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.646711    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:22:24.665489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.665289    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh" pod "d1295ab6-c34b-4fb8-acc5-89cc095986fc" (UID: "d1295ab6-c34b-4fb8-acc5-89cc095986fc"). InnerVolumeSpecName "kube-api-access-bbgmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:24.665976 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.665955    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs" pod "d1295ab6-c34b-4fb8-acc5-89cc095986fc" (UID: "d1295ab6-c34b-4fb8-acc5-89cc095986fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:24.673190 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.673166    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm" pod "d1295ab6-c34b-4fb8-acc5-89cc095986fc" (UID: "d1295ab6-c34b-4fb8-acc5-89cc095986fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.744739    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache\") pod \"7ab09e66-6512-431a-b3ab-0358b57baeac\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") "
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.745033    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache" pod "7ab09e66-6512-431a-b3ab-0358b57baeac" (UID: "7ab09e66-6512-431a-b3ab-0358b57baeac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.745279    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir\") pod \"7ab09e66-6512-431a-b3ab-0358b57baeac\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") "
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.745591    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir" pod "7ab09e66-6512-431a-b3ab-0358b57baeac" (UID: "7ab09e66-6512-431a-b3ab-0358b57baeac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.745744    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs\" (UniqueName: \"kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs\") pod \"7ab09e66-6512-431a-b3ab-0358b57baeac\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") "
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.746174    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv\" (UniqueName: \"kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv\") pod \"7ab09e66-6512-431a-b3ab-0358b57baeac\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") "
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.746607    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home\") pod \"7ab09e66-6512-431a-b3ab-0358b57baeac\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") "
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.746946    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home" pod "7ab09e66-6512-431a-b3ab-0358b57baeac" (UID: "7ab09e66-6512-431a-b3ab-0358b57baeac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747014    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm\") pod \"7ab09e66-6512-431a-b3ab-0358b57baeac\" (UID: \"7ab09e66-6512-431a-b3ab-0358b57baeac\") "
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747910    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747933    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747946    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747957    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747969    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1295ab6-c34b-4fb8-acc5-89cc095986fc-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747979    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1295ab6-c34b-4fb8-acc5-89cc095986fc-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.747995    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbgmh\" (UniqueName: \"kubernetes.io/projected/d1295ab6-c34b-4fb8-acc5-89cc095986fc-kube-api-access-bbgmh\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.748010    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.748023    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.753904    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs" pod "7ab09e66-6512-431a-b3ab-0358b57baeac" (UID: "7ab09e66-6512-431a-b3ab-0358b57baeac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:24.756932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.754843    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm" pod "7ab09e66-6512-431a-b3ab-0358b57baeac" (UID: "7ab09e66-6512-431a-b3ab-0358b57baeac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:24.773106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.771840    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv" pod "7ab09e66-6512-431a-b3ab-0358b57baeac" (UID: "7ab09e66-6512-431a-b3ab-0358b57baeac"). InnerVolumeSpecName "kube-api-access-mbwnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:24.850235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.850080    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ab09e66-6512-431a-b3ab-0358b57baeac-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.850235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.850134    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbwnv\" (UniqueName: \"kubernetes.io/projected/7ab09e66-6512-431a-b3ab-0358b57baeac-kube-api-access-mbwnv\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.850235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.850152    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ab09e66-6512-431a-b3ab-0358b57baeac-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:24.911855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.911016    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:24.929515 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:24.928703    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:25.120856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.120761    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" event={"ID":"7ab09e66-6512-431a-b3ab-0358b57baeac","Type":"ContainerDied","Data":"53e599f1bfab8f47ff500d646ae99edd12328c5eefedc041cf6c1d065d7555df"}
Oct 30 20:22:25.120856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.120845    6389 scope.go:122] "RemoveContainer" containerID="296c7e28e24a941ad5e2e6f39dc419c2b965e848bf195bf43351093486bacc2a"
Oct 30 20:22:25.121091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.121072    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"
Oct 30 20:22:25.158427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.158296    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8" event={"ID":"d1295ab6-c34b-4fb8-acc5-89cc095986fc","Type":"ContainerDied","Data":"acc33d6834f36e4d794a33a355ca96d325715df74621ccb94b5ad9d7e7e1d48b"}
Oct 30 20:22:25.158427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.158352    6389 scope.go:122] "RemoveContainer" containerID="cfc26422c275c243022694e9fdbd6b0ab70c823e690e1a3680969c1c94577951"
Oct 30 20:22:25.158617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.158590    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"
Oct 30 20:22:25.205154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.205110    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"]
Oct 30 20:22:25.213041 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.213006    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/distinct-w6-75e734-kserve-5688b6dc96-8gps8"]
Oct 30 20:22:25.234687 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.234661    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"]
Oct 30 20:22:25.238974 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.238949    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2"]
Oct 30 20:22:25.266422 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.265456    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:25.271496 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.271257    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:25.298701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.298659    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-75e734/premium-w6-75e734-kserve-6665fdf8c7-fprn2" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:25.358243 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.358211    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:25.373409 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.373304    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" containerID="cri-o://ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73" gracePeriod=45
Oct 30 20:22:25.394456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.394422    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": dial tcp 10.134.0.61:8000: connect: connection refused"
Oct 30 20:22:25.401206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.401173    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:25.402361 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.402329    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:22:25.463601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.463568    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:25.465657 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.465633    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:22:25.498203 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.496806    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:25.718819 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.718787    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:22:25.795544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.795508    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs\" (UniqueName: \"kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs\") pod \"c447eceb-5c07-494f-872e-979635d9fd83\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") "
Oct 30 20:22:25.795774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.795739    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache\") pod \"c447eceb-5c07-494f-872e-979635d9fd83\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") "
Oct 30 20:22:25.796218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.796181    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache" pod "c447eceb-5c07-494f-872e-979635d9fd83" (UID: "c447eceb-5c07-494f-872e-979635d9fd83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:25.796435 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.796411    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:25.806749 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:25.806716    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:22:25.808341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.807851    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs" pod "c447eceb-5c07-494f-872e-979635d9fd83" (UID: "c447eceb-5c07-494f-872e-979635d9fd83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:25.897664 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.897635    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home\") pod \"c447eceb-5c07-494f-872e-979635d9fd83\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") "
Oct 30 20:22:25.897836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.897737    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm\") pod \"c447eceb-5c07-494f-872e-979635d9fd83\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") "
Oct 30 20:22:25.897836 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.897826    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir\") pod \"c447eceb-5c07-494f-872e-979635d9fd83\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") "
Oct 30 20:22:25.897908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.897892    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home" pod "c447eceb-5c07-494f-872e-979635d9fd83" (UID: "c447eceb-5c07-494f-872e-979635d9fd83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:25.897945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.897918    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4\" (UniqueName: \"kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4\") pod \"c447eceb-5c07-494f-872e-979635d9fd83\" (UID: \"c447eceb-5c07-494f-872e-979635d9fd83\") "
Oct 30 20:22:25.898108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.898089    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir" pod "c447eceb-5c07-494f-872e-979635d9fd83" (UID: "c447eceb-5c07-494f-872e-979635d9fd83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:25.898493 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.898466    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c447eceb-5c07-494f-872e-979635d9fd83-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:25.898493 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.898489    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:25.898690 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.898501    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:25.899929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.899906    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm" pod "c447eceb-5c07-494f-872e-979635d9fd83" (UID: "c447eceb-5c07-494f-872e-979635d9fd83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:25.899997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.899961    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4" pod "c447eceb-5c07-494f-872e-979635d9fd83" (UID: "c447eceb-5c07-494f-872e-979635d9fd83"). InnerVolumeSpecName "kube-api-access-89gs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:25.916755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.916735    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" path="/var/lib/kubelet/pods/7ab09e66-6512-431a-b3ab-0358b57baeac/volumes"
Oct 30 20:22:25.917175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.917161    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" path="/var/lib/kubelet/pods/d1295ab6-c34b-4fb8-acc5-89cc095986fc/volumes"
Oct 30 20:22:25.999751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.999713    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89gs4\" (UniqueName: \"kubernetes.io/projected/c447eceb-5c07-494f-872e-979635d9fd83-kube-api-access-89gs4\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:25.999751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:25.999744    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c447eceb-5c07-494f-872e-979635d9fd83-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:26.022018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.021978    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing" containerID="cri-o://8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef" gracePeriod=25
Oct 30 20:22:26.091962 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.091922    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"]
Oct 30 20:22:26.092601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092584    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main"
Oct 30 20:22:26.092601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092600    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092627    6389 cpu_manager.go:414] "containerMap: removing container" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092632    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092644    6389 cpu_manager.go:414] "containerMap: removing container" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092649    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092661    6389 cpu_manager.go:414] "containerMap: removing container" podUID="31c977a9-c12f-4686-9249-52dc228c2cb5" containerName="revoke-keys"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092666    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31c977a9-c12f-4686-9249-52dc228c2cb5" containerName="revoke-keys"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092685    6389 cpu_manager.go:414] "containerMap: removing container" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main"
Oct 30 20:22:26.092746 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092690    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main"
Oct 30 20:22:26.093182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092795    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ab09e66-6512-431a-b3ab-0358b57baeac" containerName="main"
Oct 30 20:22:26.093182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092805    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="1029303a-e800-4bf0-be8c-bbe05fcec90d" containerName="main"
Oct 30 20:22:26.093182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092814    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1295ab6-c34b-4fb8-acc5-89cc095986fc" containerName="main"
Oct 30 20:22:26.093182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092820    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="31c977a9-c12f-4686-9249-52dc228c2cb5" containerName="revoke-keys"
Oct 30 20:22:26.093182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.092827    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="c447eceb-5c07-494f-872e-979635d9fd83" containerName="main"
Oct 30 20:22:26.097622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.097602    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.100914 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.100884    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.101031 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.100970    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.101094 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.101068    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n62f\" (UniqueName: \"kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.116555 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.116519    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:22:26.116844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.116817    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:22:26.119600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.119580    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"]
Oct 30 20:22:26.169994 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.169961    6389 generic.go:356] "Generic (PLEG): container finished" podID="c447eceb-5c07-494f-872e-979635d9fd83" containerID="ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73" exitCode=0
Oct 30 20:22:26.170188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.170040    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"
Oct 30 20:22:26.170188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.170076    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" event={"ID":"c447eceb-5c07-494f-872e-979635d9fd83","Type":"ContainerDied","Data":"ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73"}
Oct 30 20:22:26.170188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.170109    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz" event={"ID":"c447eceb-5c07-494f-872e-979635d9fd83","Type":"ContainerDied","Data":"e27bba1a8a8fdf7425ca23db494d2777240728ae2cdb0891cca7b30a9862bd6f"}
Oct 30 20:22:26.170188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.170171    6389 scope.go:122] "RemoveContainer" containerID="ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73"
Oct 30 20:22:26.173649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.173580    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:26.182698 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.182676    6389 scope.go:122] "RemoveContainer" containerID="ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73"
Oct 30 20:22:26.183011 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:26.182988    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73\": container with ID starting with ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73 not found: ID does not exist" containerID="ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73"
Oct 30 20:22:26.183096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.183033    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73"} err="failed to get container status \"ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73\": rpc error: code = NotFound desc = could not find container \"ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73\": container with ID starting with ffce17e4bfdb4de5e73df3aa965c8c7166f48f0ee6c1b9a01d643f821c79ec73 not found: ID does not exist"
Oct 30 20:22:26.202809 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.202785    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.202941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.202865    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.202941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.202915    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n62f\" (UniqueName: \"kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.204159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.204053    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.204279 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.204233    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:26.205858 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.205833    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:26.207043 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.206677    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.224272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.224246    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n62f\" (UniqueName: \"kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f\") pod \"maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.237980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.237918    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"]
Oct 30 20:22:26.250573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.250478    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-9c901b/premium-w4-9c901b-kserve-6b78c9f4b8-hl2gz"]
Oct 30 20:22:26.280038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.280001    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:26.286507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.286473    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:26.411288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.411257    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:26.696056 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:26.696024    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"]
Oct 30 20:22:26.696210 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:26.696109    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd26faff4_0ee8_4790_ada6_d93808fe7711.slice/crio-91b2f7687ea424ad16f573da58e354bbdaf697b3f215c5c125356b8a58260f25 WatchSource:0}: Error finding container 91b2f7687ea424ad16f573da58e354bbdaf697b3f215c5c125356b8a58260f25: Status 404 returned error can't find the container with id 91b2f7687ea424ad16f573da58e354bbdaf697b3f215c5c125356b8a58260f25
Oct 30 20:22:27.182005 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:27.181971    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x" event={"ID":"d26faff4-0ee8-4790-ada6-d93808fe7711","Type":"ContainerStarted","Data":"924136df4cc59e54b1eb26a4d413bab641381e5557e0daf1064f483f8180e3db"}
Oct 30 20:22:27.182486 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:27.182024    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x" event={"ID":"d26faff4-0ee8-4790-ada6-d93808fe7711","Type":"ContainerStarted","Data":"91b2f7687ea424ad16f573da58e354bbdaf697b3f215c5c125356b8a58260f25"}
Oct 30 20:22:27.207010 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:27.206948    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x" podStartSLOduration=1.206930333 podStartE2EDuration="1.206930333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:27.200859992 +0000 UTC m=+1067.400432942" watchObservedRunningTime="2026-10-30 20:22:27.206930333 +0000 UTC m=+1067.406503261"
Oct 30 20:22:27.286940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:27.286860    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:27.794304 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:27.794265    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:27.918753 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:27.918654    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c447eceb-5c07-494f-872e-979635d9fd83" path="/var/lib/kubelet/pods/c447eceb-5c07-494f-872e-979635d9fd83/volumes"
Oct 30 20:22:28.191057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:28.190969    6389 generic.go:356] "Generic (PLEG): container finished" podID="d26faff4-0ee8-4790-ada6-d93808fe7711" containerID="924136df4cc59e54b1eb26a4d413bab641381e5557e0daf1064f483f8180e3db" exitCode=0
Oct 30 20:22:28.191464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:28.191056    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x" event={"ID":"d26faff4-0ee8-4790-ada6-d93808fe7711","Type":"ContainerDied","Data":"924136df4cc59e54b1eb26a4d413bab641381e5557e0daf1064f483f8180e3db"}
Oct 30 20:22:28.305165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:28.305133    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:28.317419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:28.317393    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:29.350744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.350718    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:29.446400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.446364    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca\") pod \"d26faff4-0ee8-4790-ada6-d93808fe7711\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") "
Oct 30 20:22:29.446400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.446409    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f\" (UniqueName: \"kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f\") pod \"d26faff4-0ee8-4790-ada6-d93808fe7711\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") "
Oct 30 20:22:29.446617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.446518    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca\") pod \"d26faff4-0ee8-4790-ada6-d93808fe7711\" (UID: \"d26faff4-0ee8-4790-ada6-d93808fe7711\") "
Oct 30 20:22:29.446756 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.446731    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca" pod "d26faff4-0ee8-4790-ada6-d93808fe7711" (UID: "d26faff4-0ee8-4790-ada6-d93808fe7711"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:29.447074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.447044    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:29.448539 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.448514    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca" pod "d26faff4-0ee8-4790-ada6-d93808fe7711" (UID: "d26faff4-0ee8-4790-ada6-d93808fe7711"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:29.448625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.448606    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f" pod "d26faff4-0ee8-4790-ada6-d93808fe7711" (UID: "d26faff4-0ee8-4790-ada6-d93808fe7711"). InnerVolumeSpecName "kube-api-access-7n62f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:29.548680 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.548642    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n62f\" (UniqueName: \"kubernetes.io/projected/d26faff4-0ee8-4790-ada6-d93808fe7711-kube-api-access-7n62f\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:29.548680 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.548674    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d26faff4-0ee8-4790-ada6-d93808fe7711-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:29.689752 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:29.689720    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:30.206963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:30.206881    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x" event={"ID":"d26faff4-0ee8-4790-ada6-d93808fe7711","Type":"ContainerDied","Data":"91b2f7687ea424ad16f573da58e354bbdaf697b3f215c5c125356b8a58260f25"}
Oct 30 20:22:30.206963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:30.206904    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"
Oct 30 20:22:30.206963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:30.206939    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91b2f7687ea424ad16f573da58e354bbdaf697b3f215c5c125356b8a58260f25"
Oct 30 20:22:30.447076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:30.447039    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"]
Oct 30 20:22:30.474288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:30.474258    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"]
Oct 30 20:22:30.584654 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:30.582165    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:31.093451 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.093427    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:22:31.172744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.172710    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4\" (UniqueName: \"kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4\") pod \"19d494e7-3de8-4426-97f0-1cb29375fdce\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") "
Oct 30 20:22:31.172896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.172776    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume\") pod \"19d494e7-3de8-4426-97f0-1cb29375fdce\" (UID: \"19d494e7-3de8-4426-97f0-1cb29375fdce\") "
Oct 30 20:22:31.173329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.173290    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume" pod "19d494e7-3de8-4426-97f0-1cb29375fdce" (UID: "19d494e7-3de8-4426-97f0-1cb29375fdce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:31.173468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.173424    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d494e7-3de8-4426-97f0-1cb29375fdce-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:31.175044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.175021    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4" pod "19d494e7-3de8-4426-97f0-1cb29375fdce" (UID: "19d494e7-3de8-4426-97f0-1cb29375fdce"). InnerVolumeSpecName "kube-api-access-qjzp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:31.191448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.191425    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:31.216813 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.216783    6389 generic.go:356] "Generic (PLEG): container finished" podID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerID="14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381" exitCode=0
Oct 30 20:22:31.216932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.216836    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"
Oct 30 20:22:31.216932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.216866    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" event={"ID":"19d494e7-3de8-4426-97f0-1cb29375fdce","Type":"ContainerDied","Data":"14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381"}
Oct 30 20:22:31.217002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.216948    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5" event={"ID":"19d494e7-3de8-4426-97f0-1cb29375fdce","Type":"ContainerDied","Data":"b0c01665f0d04d4ba22262419f256acdd78f49510259947f0f04f3fad3fb00b8"}
Oct 30 20:22:31.217002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.216972    6389 scope.go:122] "RemoveContainer" containerID="14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381"
Oct 30 20:22:31.232146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.232104    6389 scope.go:122] "RemoveContainer" containerID="14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381"
Oct 30 20:22:31.232889 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:31.232825    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381\": container with ID starting with 14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381 not found: ID does not exist" containerID="14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381"
Oct 30 20:22:31.232889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.232865    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381"} err="failed to get container status \"14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381\": rpc error: code = NotFound desc = could not find container \"14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381\": container with ID starting with 14ee35012e0303a68af3a5e40a067ab38457e3066f9d7eb98d943c9ce9d47381 not found: ID does not exist"
Oct 30 20:22:31.265066 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.265026    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"]
Oct 30 20:22:31.268902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.268867    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c646da-6c648445ff-4pcb5"]
Oct 30 20:22:31.275182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.275155    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjzp4\" (UniqueName: \"kubernetes.io/projected/19d494e7-3de8-4426-97f0-1cb29375fdce-kube-api-access-qjzp4\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:31.915663 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.915629    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" path="/var/lib/kubelet/pods/19d494e7-3de8-4426-97f0-1cb29375fdce/volumes"
Oct 30 20:22:31.925499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.925472    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"]
Oct 30 20:22:31.926019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.926007    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d26faff4-0ee8-4790-ada6-d93808fe7711" containerName="revoke-keys"
Oct 30 20:22:31.926062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.926020    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d26faff4-0ee8-4790-ada6-d93808fe7711" containerName="revoke-keys"
Oct 30 20:22:31.926062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.926041    6389 cpu_manager.go:414] "containerMap: removing container" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerName="payload-pre-processing"
Oct 30 20:22:31.926062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.926046    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerName="payload-pre-processing"
Oct 30 20:22:31.926226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.926161    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="19d494e7-3de8-4426-97f0-1cb29375fdce" containerName="payload-pre-processing"
Oct 30 20:22:31.926226 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.926175    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d26faff4-0ee8-4790-ada6-d93808fe7711" containerName="revoke-keys"
Oct 30 20:22:31.930944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.930925    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:31.941674 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.941344    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:22:31.941674 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.941426    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:22:31.942436 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.942417    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"]
Oct 30 20:22:31.977542 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.977514    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"]
Oct 30 20:22:31.977936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.977902    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" podUID="3c749eb2-04b4-4110-8367-c6e29548ec55" containerName="maas-api" containerID="cri-o://7827116243c22279e5d72a569c9b08479a8328181610bf43161d9adf91c789d2" gracePeriod=30
Oct 30 20:22:31.986695 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.986656    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:31.987078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.987052    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:31.987220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:31.987191    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86lrp\" (UniqueName: \"kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.019075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.019044    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:32.088193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.088159    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.088363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.088219    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86lrp\" (UniqueName: \"kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.088363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.088321    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.089694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.089658    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.090902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.090880    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.101897 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.101868    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86lrp\" (UniqueName: \"kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp\") pod \"maas-api-revoke-sub-01e97a29-jnppp\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.203813 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.203662    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:32.231182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.230980    6389 generic.go:356] "Generic (PLEG): container finished" podID="3c749eb2-04b4-4110-8367-c6e29548ec55" containerID="7827116243c22279e5d72a569c9b08479a8328181610bf43161d9adf91c789d2" exitCode=0
Oct 30 20:22:32.231480 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.231399    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" event={"ID":"3c749eb2-04b4-4110-8367-c6e29548ec55","Type":"ContainerDied","Data":"7827116243c22279e5d72a569c9b08479a8328181610bf43161d9adf91c789d2"}
Oct 30 20:22:32.250988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.250964    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:32.291208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.291178    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:22:32.314233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.314152    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:32.393104 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.393066    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca\") pod \"3c749eb2-04b4-4110-8367-c6e29548ec55\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") "
Oct 30 20:22:32.393309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.393136    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj\" (UniqueName: \"kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj\") pod \"3c749eb2-04b4-4110-8367-c6e29548ec55\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") "
Oct 30 20:22:32.393309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.393231    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs\") pod \"3c749eb2-04b4-4110-8367-c6e29548ec55\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") "
Oct 30 20:22:32.393492 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.393470    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\") pod \"3c749eb2-04b4-4110-8367-c6e29548ec55\" (UID: \"3c749eb2-04b4-4110-8367-c6e29548ec55\") "
Oct 30 20:22:32.393895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.393862    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca" pod "3c749eb2-04b4-4110-8367-c6e29548ec55" (UID: "3c749eb2-04b4-4110-8367-c6e29548ec55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:32.394221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.394197    6389 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c749eb2-04b4-4110-8367-c6e29548ec55-openshift-service-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:32.395926 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.395886    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs" pod "3c749eb2-04b4-4110-8367-c6e29548ec55" (UID: "3c749eb2-04b4-4110-8367-c6e29548ec55"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:32.395926 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.395901    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj" pod "3c749eb2-04b4-4110-8367-c6e29548ec55" (UID: "3c749eb2-04b4-4110-8367-c6e29548ec55"). InnerVolumeSpecName "kube-api-access-bnlfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:32.396351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.396333    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls" pod "3c749eb2-04b4-4110-8367-c6e29548ec55" (UID: "3c749eb2-04b4-4110-8367-c6e29548ec55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:32.495635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.495596    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnlfj\" (UniqueName: \"kubernetes.io/projected/3c749eb2-04b4-4110-8367-c6e29548ec55-kube-api-access-bnlfj\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:32.495635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.495634    6389 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-metrics-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:32.495635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.495644    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c749eb2-04b4-4110-8367-c6e29548ec55-maas-api-tls\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:32.550088 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.550053    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"]
Oct 30 20:22:32.555608 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:32.555563    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e30a0cb_5000_48c6_9a9a_f53b2a250d9b.slice/crio-d21f6a1a8bf3fba755707b1cc49824de8e83cb06fc8b9c408036952dd5aea828 WatchSource:0}: Error finding container d21f6a1a8bf3fba755707b1cc49824de8e83cb06fc8b9c408036952dd5aea828: Status 404 returned error can't find the container with id d21f6a1a8bf3fba755707b1cc49824de8e83cb06fc8b9c408036952dd5aea828
Oct 30 20:22:32.844146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:32.839859    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:33.267168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.263975    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv" event={"ID":"3c749eb2-04b4-4110-8367-c6e29548ec55","Type":"ContainerDied","Data":"28ca5179c90b0573561f9b20d8c936f8d125a55eb49b17ac9784e8f9aa262fb4"}
Oct 30 20:22:33.267168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.264047    6389 scope.go:122] "RemoveContainer" containerID="7827116243c22279e5d72a569c9b08479a8328181610bf43161d9adf91c789d2"
Oct 30 20:22:33.267168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.264237    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"
Oct 30 20:22:33.279751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.278728    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp" event={"ID":"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b","Type":"ContainerStarted","Data":"6f4484fa484c8f5e01bbddbcc01e0b071f97bdfa72de45dd92a58b28916424df"}
Oct 30 20:22:33.279751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.278794    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp" event={"ID":"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b","Type":"ContainerStarted","Data":"d21f6a1a8bf3fba755707b1cc49824de8e83cb06fc8b9c408036952dd5aea828"}
Oct 30 20:22:33.333940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.333857    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp" podStartSLOduration=2.333833946 podStartE2EDuration="2.333833946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:33.299546661 +0000 UTC m=+1073.499119590" watchObservedRunningTime="2026-10-30 20:22:33.333833946 +0000 UTC m=+1073.533406870"
Oct 30 20:22:33.403291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.389182    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"]
Oct 30 20:22:33.403291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.394395    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c646da-5b9dcd65c6-zflgv"]
Oct 30 20:22:33.511145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.497294    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" containerID="cri-o://b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133" gracePeriod=40
Oct 30 20:22:33.660460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.659218    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:33.729184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.728210    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 20:22:33.771973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.771691    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Oct 30 20:22:33.772267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:33.772238    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:34.163143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.154739    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c749eb2-04b4-4110-8367-c6e29548ec55" path="/var/lib/kubelet/pods/3c749eb2-04b4-4110-8367-c6e29548ec55/volumes"
Oct 30 20:22:34.318975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.318394    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:34.360207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.353081    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:34.394085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.393336    6389 generic.go:356] "Generic (PLEG): container finished" podID="8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" containerID="6f4484fa484c8f5e01bbddbcc01e0b071f97bdfa72de45dd92a58b28916424df" exitCode=0
Oct 30 20:22:34.394085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.393649    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp" event={"ID":"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b","Type":"ContainerDied","Data":"6f4484fa484c8f5e01bbddbcc01e0b071f97bdfa72de45dd92a58b28916424df"}
Oct 30 20:22:34.585346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.585243    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:34.660248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.660207    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 20:22:34.661207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.661181    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:22:34.742296 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.742249    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:34.755147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.743079    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:22:34.810797 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.788084    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:34.810797 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.788853    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:22:34.869199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.845838    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Oct 30 20:22:34.870070 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:34.870043    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:35.066089 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.065059    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:35.174754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.127486    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:35.355153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.354213    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:35.399235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.356666    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:35.447829 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.423573    6389 patch_prober.go:29] interesting pod/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:22:35.447829 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.423636    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:35.557148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.555468    6389 generic.go:356] "Generic (PLEG): container finished" podID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerID="2d65b71b1fe1906001302a9322a06a1012fbe0778abf3f23b7d92caeaa3a2ef4" exitCode=0
Oct 30 20:22:35.557148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.555918    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" event={"ID":"c5ad98e6-2926-4c0b-8f77-2482f90e912b","Type":"ContainerDied","Data":"2d65b71b1fe1906001302a9322a06a1012fbe0778abf3f23b7d92caeaa3a2ef4"}
Oct 30 20:22:35.602142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.589233    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:35.602142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.590805    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:22:35.673551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.668236    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:35.750270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.750223    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:35.780141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.775847    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Oct 30 20:22:35.788135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.781626    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:35.880745 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.879195    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:22:35.937102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.928794    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:22:35.937102 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:35.928881    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:36.159143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.140217    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:36.326184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.319550    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:36.363139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.361249    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:36.438161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.435905    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 20:22:36.492840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.458663    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:36.538476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.439372    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:36.538476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.537022    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:22:36.683543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.673467    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:36.706696 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.630101    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 30 20:22:36.764362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.755228    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:36.783329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.783290    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Oct 30 20:22:36.906869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.897355    6389 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": context deadline exceeded"
Oct 30 20:22:36.906869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:36.897790    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:37.153423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.153383    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:22:37.247352 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.247268    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:22:37.285256 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.285224    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:22:37.285516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.285496    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:22:37.315174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.298012    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:22:37.331773 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.331721    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:37.360682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.360423    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:22:37.381052 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.380993    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:37.438538 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.438440    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca\") pod \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") "
Oct 30 20:22:37.440873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.439894    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp\" (UniqueName: \"kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp\") pod \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") "
Oct 30 20:22:37.440873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.439947    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca\") pod \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\" (UID: \"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b\") "
Oct 30 20:22:37.446191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.443176    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca" pod "8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" (UID: "8e30a0cb-5000-48c6-9a9a-f53b2a250d9b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:37.452290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.450340    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp" pod "8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" (UID: "8e30a0cb-5000-48c6-9a9a-f53b2a250d9b"). InnerVolumeSpecName "kube-api-access-86lrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:37.454164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.452952    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca" pod "8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" (UID: "8e30a0cb-5000-48c6-9a9a-f53b2a250d9b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:37.483749 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.483687    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"]
Oct 30 20:22:37.542957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.542917    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume\") pod \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") "
Oct 30 20:22:37.543161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.543053    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m\" (UniqueName: \"kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m\") pod \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\" (UID: \"c5ad98e6-2926-4c0b-8f77-2482f90e912b\") "
Oct 30 20:22:37.544099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.543947    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86lrp\" (UniqueName: \"kubernetes.io/projected/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-kube-api-access-86lrp\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.544099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.543976    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.544099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.543992    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.544586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.544560    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume" pod "c5ad98e6-2926-4c0b-8f77-2482f90e912b" (UID: "c5ad98e6-2926-4c0b-8f77-2482f90e912b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:37.547525 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.547496    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m" pod "c5ad98e6-2926-4c0b-8f77-2482f90e912b" (UID: "c5ad98e6-2926-4c0b-8f77-2482f90e912b"). InnerVolumeSpecName "kube-api-access-lhx8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:37.604196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.604167    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:37.609506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.609477    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.645311    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home\") pod \"befeed63-0326-4723-9bfa-0eb4535aaae1\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") "
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.645367    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs\" (UniqueName: \"kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs\") pod \"befeed63-0326-4723-9bfa-0eb4535aaae1\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") "
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.645742    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home" pod "befeed63-0326-4723-9bfa-0eb4535aaae1" (UID: "befeed63-0326-4723-9bfa-0eb4535aaae1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.645830    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7\" (UniqueName: \"kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7\") pod \"befeed63-0326-4723-9bfa-0eb4535aaae1\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") "
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.645880    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir\") pod \"befeed63-0326-4723-9bfa-0eb4535aaae1\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") "
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.645924    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm\") pod \"befeed63-0326-4723-9bfa-0eb4535aaae1\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") "
Oct 30 20:22:37.646592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.646044    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache\") pod \"befeed63-0326-4723-9bfa-0eb4535aaae1\" (UID: \"befeed63-0326-4723-9bfa-0eb4535aaae1\") "
Oct 30 20:22:37.647044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.646636    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5ad98e6-2926-4c0b-8f77-2482f90e912b-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.647044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.646660    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.647044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.646675    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhx8m\" (UniqueName: \"kubernetes.io/projected/c5ad98e6-2926-4c0b-8f77-2482f90e912b-kube-api-access-lhx8m\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.648707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.647476    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir" pod "befeed63-0326-4723-9bfa-0eb4535aaae1" (UID: "befeed63-0326-4723-9bfa-0eb4535aaae1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:37.648707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.648036    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache" pod "befeed63-0326-4723-9bfa-0eb4535aaae1" (UID: "befeed63-0326-4723-9bfa-0eb4535aaae1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:37.650494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.650454    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7" pod "befeed63-0326-4723-9bfa-0eb4535aaae1" (UID: "befeed63-0326-4723-9bfa-0eb4535aaae1"). InnerVolumeSpecName "kube-api-access-wfqv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:37.651497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.651472    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm" pod "befeed63-0326-4723-9bfa-0eb4535aaae1" (UID: "befeed63-0326-4723-9bfa-0eb4535aaae1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:37.652551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.652523    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs" pod "befeed63-0326-4723-9bfa-0eb4535aaae1" (UID: "befeed63-0326-4723-9bfa-0eb4535aaae1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:37.707549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.707522    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:37.748467 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.748434    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befeed63-0326-4723-9bfa-0eb4535aaae1-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.748467 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.748468    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfqv7\" (UniqueName: \"kubernetes.io/projected/befeed63-0326-4723-9bfa-0eb4535aaae1-kube-api-access-wfqv7\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.748632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.748484    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.748632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.748496    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.748632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.748508    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befeed63-0326-4723-9bfa-0eb4535aaae1-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:37.777405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.776719    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" event={"ID":"c5ad98e6-2926-4c0b-8f77-2482f90e912b","Type":"ContainerDied","Data":"a8dc28c57e3061d5e6c388d9f254fc3d521e5211c8dced8a5bcd17be56f6f9fd"}
Oct 30 20:22:37.777405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.776799    6389 scope.go:122] "RemoveContainer" containerID="2d65b71b1fe1906001302a9322a06a1012fbe0778abf3f23b7d92caeaa3a2ef4"
Oct 30 20:22:37.777405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.776963    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"
Oct 30 20:22:37.810171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.805738    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp" event={"ID":"8e30a0cb-5000-48c6-9a9a-f53b2a250d9b","Type":"ContainerDied","Data":"d21f6a1a8bf3fba755707b1cc49824de8e83cb06fc8b9c408036952dd5aea828"}
Oct 30 20:22:37.810171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.805802    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d21f6a1a8bf3fba755707b1cc49824de8e83cb06fc8b9c408036952dd5aea828"
Oct 30 20:22:37.810171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.805887    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"
Oct 30 20:22:37.817290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.816042    6389 generic.go:356] "Generic (PLEG): container finished" podID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerID="b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133" exitCode=0
Oct 30 20:22:37.817290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.816224    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" event={"ID":"befeed63-0326-4723-9bfa-0eb4535aaae1","Type":"ContainerDied","Data":"b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133"}
Oct 30 20:22:37.817290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.816263    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96" event={"ID":"befeed63-0326-4723-9bfa-0eb4535aaae1","Type":"ContainerDied","Data":"bf7e37de848abbfb7fddf6bcdad86c0fbd235572da7465a4459665a1faf3f8e9"}
Oct 30 20:22:37.817290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.816334    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"
Oct 30 20:22:37.865283 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.865256    6389 scope.go:122] "RemoveContainer" containerID="b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133"
Oct 30 20:22:37.878057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.878036    6389 scope.go:122] "RemoveContainer" containerID="b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133"
Oct 30 20:22:37.880619 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:37.880594    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133\": container with ID starting with b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133 not found: ID does not exist" containerID="b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133"
Oct 30 20:22:37.880798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.880767    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133"} err="failed to get container status \"b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133\": rpc error: code = NotFound desc = could not find container \"b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133\": container with ID starting with b9aad5034204b56df674b8be561c8d7844876db6fe134a2b80da73c6156fe133 not found: ID does not exist"
Oct 30 20:22:37.885369 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.885348    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"]
Oct 30 20:22:37.891786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.891762    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-44135b/simulator-w5-44135b-kserve-5fcb85c46f-8gc96"]
Oct 30 20:22:37.898939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.898914    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"]
Oct 30 20:22:37.904929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.904906    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc"]
Oct 30 20:22:37.917518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.917492    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" path="/var/lib/kubelet/pods/befeed63-0326-4723-9bfa-0eb4535aaae1/volumes"
Oct 30 20:22:37.917932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:37.917918    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" path="/var/lib/kubelet/pods/c5ad98e6-2926-4c0b-8f77-2482f90e912b/volumes"
Oct 30 20:22:38.289844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:38.289802    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-75e734-67758b9799-cq2lc" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.50:50052\" within 1s: context deadline exceeded"
Oct 30 20:22:38.290056 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:38.289917    6389 logging.go:55] [core] [Channel #765 SubChannel #766]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.50:50052", ServerName: "10.134.0.50:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.50:50052: operation was canceled"
Oct 30 20:22:38.843149 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:38.843086    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:38.946366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:38.946330    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:39.074797 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:39.074760    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:39.372269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:39.372212    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"]
Oct 30 20:22:39.395191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:39.395161    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-44135b-66mdd"]
Oct 30 20:22:39.511459 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:39.511425    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:39.790194 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:39.790165    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:39.916290 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:39.916257    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31c977a9-c12f-4686-9249-52dc228c2cb5" path="/var/lib/kubelet/pods/31c977a9-c12f-4686-9249-52dc228c2cb5/volumes"
Oct 30 20:22:40.807459 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:40.807422    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:40.908488 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:40.908450    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:41.010060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.010025    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:41.266300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.266271    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"]
Oct 30 20:22:41.266564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.266490    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy" containerID="cri-o://054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c" gracePeriod=30
Oct 30 20:22:41.279986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.270339    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"]
Oct 30 20:22:41.279986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.270512    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy" containerID="cri-o://50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b" gracePeriod=30
Oct 30 20:22:41.287377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.287341    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"]
Oct 30 20:22:41.287688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.287624    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy" containerID="cri-o://7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903" gracePeriod=30
Oct 30 20:22:41.314152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.314105    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:41.890525 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.890489    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:41.994622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.994591    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"]
Oct 30 20:22:41.995309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995282    6389 cpu_manager.go:414] "containerMap: removing container" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing"
Oct 30 20:22:41.995309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995306    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995335    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3c749eb2-04b4-4110-8367-c6e29548ec55" containerName="maas-api"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995342    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c749eb2-04b4-4110-8367-c6e29548ec55" containerName="maas-api"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995394    6389 cpu_manager.go:414] "containerMap: removing container" podUID="8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" containerName="revoke-keys"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995401    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" containerName="revoke-keys"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995416    6389 cpu_manager.go:414] "containerMap: removing container" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995421    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main"
Oct 30 20:22:41.995535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995535    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5ad98e6-2926-4c0b-8f77-2482f90e912b" containerName="payload-pre-processing"
Oct 30 20:22:41.995865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995550    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c749eb2-04b4-4110-8367-c6e29548ec55" containerName="maas-api"
Oct 30 20:22:41.995865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995560    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" containerName="revoke-keys"
Oct 30 20:22:41.995865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:41.995571    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="befeed63-0326-4723-9bfa-0eb4535aaae1" containerName="main"
Oct 30 20:22:42.001687 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.001657    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.020129 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.020086    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"]
Oct 30 20:22:42.054501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.054463    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:22:42.054701 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.054599    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:22:42.102501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.102464    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:42.162269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.162177    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.162618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.162594    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.170165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.166369    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdxnp\" (UniqueName: \"kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.256619 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.256567    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:42.268218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.268177    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdxnp\" (UniqueName: \"kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.268386 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.268344    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.268488 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.268470    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.269556 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.269529    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.278252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.276910    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.281932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.281901    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdxnp\" (UniqueName: \"kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp\") pod \"maas-api-revoke-sub-cc94a4fb-ndx7r\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.326106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.326075    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:42.484659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.484615    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerName="payload-pre-processing" containerID="cri-o://f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb" gracePeriod=25
Oct 30 20:22:42.541340 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.541309    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:42.620310 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.620277    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"]
Oct 30 20:22:42.621095 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:42.621070    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a73c0a_3dfb_4bdc_8c4a_3489b4273ccf.slice/crio-cedfd1c972e219ff89a62655109491322912a4db00f63bd938626198580dbdb4 WatchSource:0}: Error finding container cedfd1c972e219ff89a62655109491322912a4db00f63bd938626198580dbdb4: Status 404 returned error can't find the container with id cedfd1c972e219ff89a62655109491322912a4db00f63bd938626198580dbdb4
Oct 30 20:22:42.848390 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.848310    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:42.863727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:42.863698    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r" event={"ID":"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf","Type":"ContainerStarted","Data":"cedfd1c972e219ff89a62655109491322912a4db00f63bd938626198580dbdb4"}
Oct 30 20:22:43.424914 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:43.424882    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:43.529921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:43.529889    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:43.871871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:43.871824    6389 generic.go:356] "Generic (PLEG): container finished" podID="95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" containerID="456f6d86f3c565036f8dc25ad491f9b60afdab8a249a0d5eee8bfc6e6d3afae0" exitCode=0
Oct 30 20:22:43.872103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:43.871919    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r" event={"ID":"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf","Type":"ContainerDied","Data":"456f6d86f3c565036f8dc25ad491f9b60afdab8a249a0d5eee8bfc6e6d3afae0"}
Oct 30 20:22:44.342998 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:44.342964    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:45.007231 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.007085    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:45.119112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.119084    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:45.187199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.185060    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:45.216138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.208524    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca\") pod \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") "
Oct 30 20:22:45.216138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.208792    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca\") pod \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") "
Oct 30 20:22:45.216138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.208969    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp\" (UniqueName: \"kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp\") pod \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\" (UID: \"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf\") "
Oct 30 20:22:45.232165 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.232061    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca" pod "95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" (UID: "95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:45.239168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.236137    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca" pod "95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" (UID: "95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:45.293901 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.278354    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp" pod "95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" (UID: "95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf"). InnerVolumeSpecName "kube-api-access-vdxnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:45.312791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.297052    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:45.312791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.311188    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:45.312791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.311226    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdxnp\" (UniqueName: \"kubernetes.io/projected/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-kube-api-access-vdxnp\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:45.312791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.311242    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:45.458571 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.458496    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:45.468601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.468565    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" containerID="cri-o://c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d" gracePeriod=45
Oct 30 20:22:45.496333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.496296    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main" containerID="cri-o://07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea" gracePeriod=45
Oct 30 20:22:45.727844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.727799    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": dial tcp 10.134.0.68:8000: connect: connection refused"
Oct 30 20:22:45.802557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.802467    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:45.851474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.851447    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:22:45.894313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.894275    6389 generic.go:356] "Generic (PLEG): container finished" podID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerID="07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea" exitCode=0
Oct 30 20:22:45.894499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.894385    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" event={"ID":"3d9bb5a6-2863-4b96-92d0-5b5579f20074","Type":"ContainerDied","Data":"07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea"}
Oct 30 20:22:45.894499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.894428    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m" event={"ID":"3d9bb5a6-2863-4b96-92d0-5b5579f20074","Type":"ContainerDied","Data":"019c84573f40517b07f2326060db32898a69a707fa9a61ec3b4994ffd3cf85e1"}
Oct 30 20:22:45.894499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.894453    6389 scope.go:122] "RemoveContainer" containerID="07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea"
Oct 30 20:22:45.894668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.894611    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"
Oct 30 20:22:45.899559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.899532    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"
Oct 30 20:22:45.899768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.899738    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r" event={"ID":"95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf","Type":"ContainerDied","Data":"cedfd1c972e219ff89a62655109491322912a4db00f63bd938626198580dbdb4"}
Oct 30 20:22:45.899862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.899809    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cedfd1c972e219ff89a62655109491322912a4db00f63bd938626198580dbdb4"
Oct 30 20:22:45.901539 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.901508    6389 generic.go:356] "Generic (PLEG): container finished" podID="84969108-4995-4703-b73f-07edbff07dc5" containerID="c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d" exitCode=0
Oct 30 20:22:45.901665 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.901584    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"
Oct 30 20:22:45.901760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.901718    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" event={"ID":"84969108-4995-4703-b73f-07edbff07dc5","Type":"ContainerDied","Data":"c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d"}
Oct 30 20:22:45.901816 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.901781    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w" event={"ID":"84969108-4995-4703-b73f-07edbff07dc5","Type":"ContainerDied","Data":"e4ec57d6468f76740d655db69979a4f2c71abbef4f992691032887db49b34c3a"}
Oct 30 20:22:45.908570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.908542    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:45.911249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.911228    6389 scope.go:122] "RemoveContainer" containerID="07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea"
Oct 30 20:22:45.912754 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:45.912723    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea\": container with ID starting with 07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea not found: ID does not exist" containerID="07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea"
Oct 30 20:22:45.912864 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.912763    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea"} err="failed to get container status \"07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea\": rpc error: code = NotFound desc = could not find container \"07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea\": container with ID starting with 07bebb3efc7ae0a0bded9024ee414c7ef7343fa8427caf470e8bbb5f5458b0ea not found: ID does not exist"
Oct 30 20:22:45.912864 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.912785    6389 scope.go:122] "RemoveContainer" containerID="c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d"
Oct 30 20:22:45.924543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924520    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home\") pod \"84969108-4995-4703-b73f-07edbff07dc5\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") "
Oct 30 20:22:45.924663 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924593    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm\") pod \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") "
Oct 30 20:22:45.924726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924678    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs\" (UniqueName: \"kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs\") pod \"84969108-4995-4703-b73f-07edbff07dc5\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") "
Oct 30 20:22:45.924726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924710    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir\") pod \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") "
Oct 30 20:22:45.924835 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924773    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home\") pod \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") "
Oct 30 20:22:45.924890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924872    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm\" (UniqueName: \"kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm\") pod \"84969108-4995-4703-b73f-07edbff07dc5\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") "
Oct 30 20:22:45.924949 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924905    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs\") pod \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") "
Oct 30 20:22:45.925007 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.924949    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache\") pod \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") "
Oct 30 20:22:45.925062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.925004    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm\") pod \"84969108-4995-4703-b73f-07edbff07dc5\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") "
Oct 30 20:22:45.925155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.925071    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache\") pod \"84969108-4995-4703-b73f-07edbff07dc5\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") "
Oct 30 20:22:45.925219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.925152    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt\" (UniqueName: \"kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt\") pod \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\" (UID: \"3d9bb5a6-2863-4b96-92d0-5b5579f20074\") "
Oct 30 20:22:45.925281 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.925240    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir\") pod \"84969108-4995-4703-b73f-07edbff07dc5\" (UID: \"84969108-4995-4703-b73f-07edbff07dc5\") "
Oct 30 20:22:45.926254 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.926225    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir" pod "84969108-4995-4703-b73f-07edbff07dc5" (UID: "84969108-4995-4703-b73f-07edbff07dc5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.928183 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.928059    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir" pod "3d9bb5a6-2863-4b96-92d0-5b5579f20074" (UID: "3d9bb5a6-2863-4b96-92d0-5b5579f20074"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.928653 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.928466    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home" pod "3d9bb5a6-2863-4b96-92d0-5b5579f20074" (UID: "3d9bb5a6-2863-4b96-92d0-5b5579f20074"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.928955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.928779    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache" pod "3d9bb5a6-2863-4b96-92d0-5b5579f20074" (UID: "3d9bb5a6-2863-4b96-92d0-5b5579f20074"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.929113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.929087    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home" pod "84969108-4995-4703-b73f-07edbff07dc5" (UID: "84969108-4995-4703-b73f-07edbff07dc5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.929763 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.929744    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache" pod "84969108-4995-4703-b73f-07edbff07dc5" (UID: "84969108-4995-4703-b73f-07edbff07dc5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.929929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.929910    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs" pod "3d9bb5a6-2863-4b96-92d0-5b5579f20074" (UID: "3d9bb5a6-2863-4b96-92d0-5b5579f20074"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:45.937331 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.937086    6389 scope.go:122] "RemoveContainer" containerID="c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d"
Oct 30 20:22:45.937677 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:45.937509    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d\": container with ID starting with c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d not found: ID does not exist" containerID="c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d"
Oct 30 20:22:45.937677 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.937543    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d"} err="failed to get container status \"c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d\": rpc error: code = NotFound desc = could not find container \"c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d\": container with ID starting with c3d141f577c3e6b1fcda7d78b236dfb93b7b4537aff0a2e45f8c16b71973f33d not found: ID does not exist"
Oct 30 20:22:45.943537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.943488    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm" pod "3d9bb5a6-2863-4b96-92d0-5b5579f20074" (UID: "3d9bb5a6-2863-4b96-92d0-5b5579f20074"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:45.943635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.943540    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs" pod "84969108-4995-4703-b73f-07edbff07dc5" (UID: "84969108-4995-4703-b73f-07edbff07dc5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:45.943635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.943548    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt" pod "3d9bb5a6-2863-4b96-92d0-5b5579f20074" (UID: "3d9bb5a6-2863-4b96-92d0-5b5579f20074"). InnerVolumeSpecName "kube-api-access-rbddt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:45.943635 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.943587    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm" pod "84969108-4995-4703-b73f-07edbff07dc5" (UID: "84969108-4995-4703-b73f-07edbff07dc5"). InnerVolumeSpecName "kube-api-access-9xjnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:45.945111 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:45.945089    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm" pod "84969108-4995-4703-b73f-07edbff07dc5" (UID: "84969108-4995-4703-b73f-07edbff07dc5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027536    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027566    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027575    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84969108-4995-4703-b73f-07edbff07dc5-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027583    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027592    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027600    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xjnm\" (UniqueName: \"kubernetes.io/projected/84969108-4995-4703-b73f-07edbff07dc5-kube-api-access-9xjnm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027609    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9bb5a6-2863-4b96-92d0-5b5579f20074-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027618    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9bb5a6-2863-4b96-92d0-5b5579f20074-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027628    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.027629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027636    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.028263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027644    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbddt\" (UniqueName: \"kubernetes.io/projected/3d9bb5a6-2863-4b96-92d0-5b5579f20074-kube-api-access-rbddt\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.028263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.027656    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84969108-4995-4703-b73f-07edbff07dc5-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.236748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.236697    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"]
Oct 30 20:22:46.242140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.242098    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-909519/simulator-w2-909519-kserve-86fb7498b-mp65m"]
Oct 30 20:22:46.254217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.254178    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"]
Oct 30 20:22:46.274769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.274483    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-909519/distinct-2-w2-909519-kserve-75bd58ffc7-7vx2w"]
Oct 30 20:22:46.281601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.281495    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:46.459516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.459393    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:46.477868 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.477840    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:22:46.484030 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.484001    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:22:46.571252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.571222    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:46.626523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.626498    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:22:46.641493 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641459    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.641648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641516    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.641648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641576    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm\" (UniqueName: \"kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm\") pod \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") "
Oct 30 20:22:46.641766 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641649    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.641766 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641703    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.641872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641762    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.641872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641800    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.641872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641832    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.642025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641870    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.642025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641899    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.642025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641936    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.642025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.641981    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.642025 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642016    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.642373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642077    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.642373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642143    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume\") pod \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\" (UID: \"d6747c8e-cf63-4849-aeee-dbbbcc43ef09\") "
Oct 30 20:22:46.642373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642182    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.642373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642240    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.642373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642310    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo\") pod \"680980fd-6738-4355-b88e-d413ed547214\" (UID: \"680980fd-6738-4355-b88e-d413ed547214\") "
Oct 30 20:22:46.642602 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642384    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.642602 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.642443    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8\") pod \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\" (UID: \"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c\") "
Oct 30 20:22:46.644139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.643675    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.644743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.644540    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:46.645240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.645219    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume" pod "d6747c8e-cf63-4849-aeee-dbbbcc43ef09" (UID: "d6747c8e-cf63-4849-aeee-dbbbcc43ef09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:46.645360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.645214    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.645749 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.645726    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.646108 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.645980    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:46.646846 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.646537    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "kube-api-access-j5qt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.646846 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.646570    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.646846 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.646804    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.647048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.646970    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.647192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.647169    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.647329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.647254    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.649645 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.649582    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:46.651180 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.651142    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.651911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.651890    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm" pod "d6747c8e-cf63-4849-aeee-dbbbcc43ef09" (UID: "d6747c8e-cf63-4849-aeee-dbbbcc43ef09"). InnerVolumeSpecName "kube-api-access-cmkbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.656371 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.656336    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.656532 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.656504    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.660808 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.660772    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token" pod "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" (UID: "e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.666855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.665483    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "kube-api-access-xwbtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.673159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.671644    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo" pod "680980fd-6738-4355-b88e-d413ed547214" (UID: "680980fd-6738-4355-b88e-d413ed547214"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:46.743562 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.743519    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.743752 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.743598    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.743752 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.743707    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.744020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.743758    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.744020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.743791    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.744432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.744007    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:46.744432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.744369    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.744432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.744429    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.744620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.744496    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.744620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.744585    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token\") pod \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\" (UID: \"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c\") "
Oct 30 20:22:46.745250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745104    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745262    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745280    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745294    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745319    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745334    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745347    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745361    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745358    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745382    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwbtz\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-kube-api-access-xwbtz\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745396    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745410    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680980fd-6738-4355-b88e-d413ed547214-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745462    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745478    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5qt8\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-kube-api-access-j5qt8\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745493    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745507    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745526    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmkbm\" (UniqueName: \"kubernetes.io/projected/d6747c8e-cf63-4849-aeee-dbbbcc43ef09-kube-api-access-cmkbm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745541    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680980fd-6738-4355-b88e-d413ed547214-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745551    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745555    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745582    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680980fd-6738-4355-b88e-d413ed547214-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745599    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680980fd-6738-4355-b88e-d413ed547214-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.745739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745615    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.746276 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.745979    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.746603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.746570    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.748161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.748102    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:46.748357 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.748332    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.748608 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.748586    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "kube-api-access-vdkr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:46.748737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.748717    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo" pod "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" (UID: "ea8f5a6d-cd2b-42b0-8020-5b805f498a5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847513    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847545    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdkr9\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-kube-api-access-vdkr9\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847556    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847564    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847573    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847582    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847590    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.847597 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.847598    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:46.878089 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.878053    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:46.910519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.910481    6389 generic.go:356] "Generic (PLEG): container finished" podID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerID="50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b" exitCode=0
Oct 30 20:22:46.910691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.910548    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"
Oct 30 20:22:46.910691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.910569    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" event={"ID":"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c","Type":"ContainerDied","Data":"50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b"}
Oct 30 20:22:46.910691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.910628    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6" event={"ID":"e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c","Type":"ContainerDied","Data":"bf4bcfb36d4addd582688b5f56e707d68b1c43315c1bb993cbfa47b154f2cf85"}
Oct 30 20:22:46.910691 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.910661    6389 scope.go:122] "RemoveContainer" containerID="50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b"
Oct 30 20:22:46.914217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.914012    6389 generic.go:356] "Generic (PLEG): container finished" podID="680980fd-6738-4355-b88e-d413ed547214" containerID="054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c" exitCode=0
Oct 30 20:22:46.914217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.914084    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"
Oct 30 20:22:46.914217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.914137    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" event={"ID":"680980fd-6738-4355-b88e-d413ed547214","Type":"ContainerDied","Data":"054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c"}
Oct 30 20:22:46.914217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.914183    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps" event={"ID":"680980fd-6738-4355-b88e-d413ed547214","Type":"ContainerDied","Data":"ce43170cf6fdef0a01d6dccdf286165ca5193f1a582e1652336360ddd9075eba"}
Oct 30 20:22:46.916976 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.916948    6389 generic.go:356] "Generic (PLEG): container finished" podID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerID="8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef" exitCode=0
Oct 30 20:22:46.917148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.917022    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" event={"ID":"d6747c8e-cf63-4849-aeee-dbbbcc43ef09","Type":"ContainerDied","Data":"8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef"}
Oct 30 20:22:46.917148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.917061    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" event={"ID":"d6747c8e-cf63-4849-aeee-dbbbcc43ef09","Type":"ContainerDied","Data":"80d57e59c6ffeda52c04f3095fbb413aac389540dc5906390121204bad89d0c4"}
Oct 30 20:22:46.917266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.917177    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"
Oct 30 20:22:46.922345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.922320    6389 generic.go:356] "Generic (PLEG): container finished" podID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerID="7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903" exitCode=0
Oct 30 20:22:46.922461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.922427    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" event={"ID":"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c","Type":"ContainerDied","Data":"7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903"}
Oct 30 20:22:46.922745 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.922464    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz" event={"ID":"ea8f5a6d-cd2b-42b0-8020-5b805f498a5c","Type":"ContainerDied","Data":"868ae13c3a07996314f9faa2e12398609799e705e33dc161f2a674ae2ac4cf1b"}
Oct 30 20:22:46.923373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.923327    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"
Oct 30 20:22:46.935060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.935003    6389 scope.go:122] "RemoveContainer" containerID="50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b"
Oct 30 20:22:46.969392 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:46.957567    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b\": container with ID starting with 50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b not found: ID does not exist" containerID="50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b"
Oct 30 20:22:46.969392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.957606    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b"} err="failed to get container status \"50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b\": rpc error: code = NotFound desc = could not find container \"50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b\": container with ID starting with 50409934a6d8c80e84dab3cc1ed4358224731e66e3dbb28e29dd1543f17c482b not found: ID does not exist"
Oct 30 20:22:46.969392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:46.957631    6389 scope.go:122] "RemoveContainer" containerID="054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c"
Oct 30 20:22:47.007474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.007440    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"]
Oct 30 20:22:47.012059 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.012036    6389 scope.go:122] "RemoveContainer" containerID="054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c"
Oct 30 20:22:47.012506 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:47.012475    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c\": container with ID starting with 054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c not found: ID does not exist" containerID="054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c"
Oct 30 20:22:47.012682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.012652    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c"} err="failed to get container status \"054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c\": rpc error: code = NotFound desc = could not find container \"054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c\": container with ID starting with 054f33bd753b089a8c75d204170db70c0bd0eac70d63b4a011ea85a51c62c27c not found: ID does not exist"
Oct 30 20:22:47.012791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.012686    6389 scope.go:122] "RemoveContainer" containerID="8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef"
Oct 30 20:22:47.021798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.021774    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-c2hps"]
Oct 30 20:22:47.022411 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.022388    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"]
Oct 30 20:22:47.026374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.026355    6389 scope.go:122] "RemoveContainer" containerID="8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef"
Oct 30 20:22:47.026712 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:47.026693    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef\": container with ID starting with 8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef not found: ID does not exist" containerID="8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef"
Oct 30 20:22:47.026783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.026719    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef"} err="failed to get container status \"8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef\": rpc error: code = NotFound desc = could not find container \"8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef\": container with ID starting with 8ffa43b99e4798092d42a1e7f6e00fe8a7aa284863930116be2fbb70deee74ef not found: ID does not exist"
Oct 30 20:22:47.026783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.026739    6389 scope.go:122] "RemoveContainer" containerID="7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903"
Oct 30 20:22:47.029170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.029079    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-9kqq6"]
Oct 30 20:22:47.042402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.042381    6389 scope.go:122] "RemoveContainer" containerID="7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903"
Oct 30 20:22:47.042763 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:47.042741    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903\": container with ID starting with 7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903 not found: ID does not exist" containerID="7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903"
Oct 30 20:22:47.042861 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.042772    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903"} err="failed to get container status \"7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903\": rpc error: code = NotFound desc = could not find container \"7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903\": container with ID starting with 7cc1ff54d4adb288db123fb330ef25f39cc347aa03deebc4c32ac3069284b903 not found: ID does not exist"
Oct 30 20:22:47.049838 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.049809    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"]
Oct 30 20:22:47.057408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.057380    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-c646da-openshift-default-7b65dc6f7c-z8ddz"]
Oct 30 20:22:47.074497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.074460    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"]
Oct 30 20:22:47.078083 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.078055    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24"]
Oct 30 20:22:47.279350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.279246    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-44135b-7b9bcb989f-g6x24" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.46:50052\" within 1s: context deadline exceeded"
Oct 30 20:22:47.279542 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:47.279422    6389 logging.go:55] [core] [Channel #772 SubChannel #776]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.46:50052", ServerName: "10.134.0.46:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.46:50052: operation was canceled"
Oct 30 20:22:47.431407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.431305    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:47.533599 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.533547    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:47.937933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.937893    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" path="/var/lib/kubelet/pods/3d9bb5a6-2863-4b96-92d0-5b5579f20074/volumes"
Oct 30 20:22:47.938998 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.938975    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="680980fd-6738-4355-b88e-d413ed547214" path="/var/lib/kubelet/pods/680980fd-6738-4355-b88e-d413ed547214/volumes"
Oct 30 20:22:47.939658 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.939635    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84969108-4995-4703-b73f-07edbff07dc5" path="/var/lib/kubelet/pods/84969108-4995-4703-b73f-07edbff07dc5/volumes"
Oct 30 20:22:47.940348 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.940101    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" path="/var/lib/kubelet/pods/d6747c8e-cf63-4849-aeee-dbbbcc43ef09/volumes"
Oct 30 20:22:47.941369 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.941349    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" path="/var/lib/kubelet/pods/e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c/volumes"
Oct 30 20:22:47.941948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:47.941929    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" path="/var/lib/kubelet/pods/ea8f5a6d-cd2b-42b0-8020-5b805f498a5c/volumes"
Oct 30 20:22:48.098045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:48.098000    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:48.707564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:48.707529    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:48.834915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:48.834839    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:49.127297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.127259    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:49.143138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.140291    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"]
Oct 30 20:22:49.143138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.140567    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy" containerID="cri-o://ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d" gracePeriod=30
Oct 30 20:22:49.326629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.312326    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"]
Oct 30 20:22:49.326629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.312607    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" containerID="cri-o://98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693" gracePeriod=30
Oct 30 20:22:49.380667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.345148    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"]
Oct 30 20:22:49.380667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.345407    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" containerID="cri-o://187a52602b29a5c00425810c307c42111c800ed727716040bb4cf0eb9420abe4" gracePeriod=30
Oct 30 20:22:49.752143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.751635    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 30 20:22:49.752143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:49.751717    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:22:50.858240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:50.858184    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:51.187179 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:51.181345    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:51.360681 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:51.360631    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:51.563476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:51.563436    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:51.669513 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:51.669464    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:52.052494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.052443    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:52.158244 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.158211    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:52.452269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.452186    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"]
Oct 30 20:22:52.452478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.452452    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy" containerID="cri-o://11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312" gracePeriod=30
Oct 30 20:22:52.462759 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.462730    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"]
Oct 30 20:22:52.463148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.463103    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy" containerID="cri-o://f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49" gracePeriod=30
Oct 30 20:22:52.468164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.468143    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"]
Oct 30 20:22:52.468320 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.468304    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy" containerID="cri-o://60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5" gracePeriod=30
Oct 30 20:22:52.470105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.470085    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:52.696358 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.696322    6389 patch_prober.go:29] interesting pod/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:22:52.696522 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:52.696378    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:22:53.235288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:53.235252    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:53.344255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:53.344219    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:53.989697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:53.989654    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:54.093363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.093337    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:54.345349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.345313    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"]
Oct 30 20:22:54.345789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.345589    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy" containerID="cri-o://3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d" gracePeriod=30
Oct 30 20:22:54.350963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.350920    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"]
Oct 30 20:22:54.351221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.351195    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" containerID="cri-o://5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1" gracePeriod=30
Oct 30 20:22:54.375269 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.375237    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:54.557536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.557508    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:22:54.649158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649101    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649296 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649180    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649296 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649214    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649346    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649358    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.649423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649388    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649419    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.649569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649435    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649503    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649569 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649537    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649758 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649636    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.649758 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649694    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy\") pod \"0c2e82a7-b61b-452e-b956-a9173e4eef49\" (UID: \"0c2e82a7-b61b-452e-b956-a9173e4eef49\") "
Oct 30 20:22:54.649853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649748    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:54.649901 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.649872    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.650264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.650243    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.650264 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.650265    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c2e82a7-b61b-452e-b956-a9173e4eef49-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.650419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.650282    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.650419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.650295    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.650419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.650310    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.651921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.651900    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:54.651921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.651913    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "kube-api-access-pgftb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:54.652098 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.652075    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.652224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.652099    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token" pod "0c2e82a7-b61b-452e-b956-a9173e4eef49" (UID: "0c2e82a7-b61b-452e-b956-a9173e4eef49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:54.658791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.658773    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:22:54.707716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.707683    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"]
Oct 30 20:22:54.708564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708537    6389 cpu_manager.go:414] "containerMap: removing container" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy"
Oct 30 20:22:54.708564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708562    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708581    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708589    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708604    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708612    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708629    6389 cpu_manager.go:414] "containerMap: removing container" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708636    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708646    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708653    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708668    6389 cpu_manager.go:414] "containerMap: removing container" podUID="95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" containerName="revoke-keys"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708675    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" containerName="revoke-keys"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708687    6389 cpu_manager.go:414] "containerMap: removing container" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708694    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708716    6389 cpu_manager.go:414] "containerMap: removing container" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708724    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708735    6389 cpu_manager.go:414] "containerMap: removing container" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy"
Oct 30 20:22:54.708748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708741    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708881    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="680980fd-6738-4355-b88e-d413ed547214" containerName="istio-proxy"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708894    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea8f5a6d-cd2b-42b0-8020-5b805f498a5c" containerName="istio-proxy"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708905    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9e3acb7-bb0d-4a2f-995f-b0a873dd6e2c" containerName="istio-proxy"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708915    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d9bb5a6-2863-4b96-92d0-5b5579f20074" containerName="main"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708926    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6747c8e-cf63-4849-aeee-dbbbcc43ef09" containerName="payload-pre-processing"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708937    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="84969108-4995-4703-b73f-07edbff07dc5" containerName="main"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708949    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" containerName="revoke-keys"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708958    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerName="istio-proxy"
Oct 30 20:22:54.709303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.708970    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerName="istio-proxy"
Oct 30 20:22:54.712935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.712911    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.728018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.726558    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"]
Oct 30 20:22:54.751297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751269    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751441 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751325    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751440    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751503    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751589 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751544    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751597    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751651    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751706    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.751780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.751750    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data\") pod \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\" (UID: \"d9eb0447-0ceb-4ec2-b9e9-b0d094885335\") "
Oct 30 20:22:54.752518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752313    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgftb\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-kube-api-access-pgftb\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.752518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752343    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.752518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752360    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.752518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752375    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c2e82a7-b61b-452e-b956-a9173e4eef49-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.752518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752441    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.752518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752461    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.752891 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752594    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.752891 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752697    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.752891 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.752787    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:54.753607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.753586    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:54.753799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.753783    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:54.754450 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.754427    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:54.754903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.754881    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9" pod "d9eb0447-0ceb-4ec2-b9e9-b0d094885335" (UID: "d9eb0447-0ceb-4ec2-b9e9-b0d094885335"). InnerVolumeSpecName "kube-api-access-gcbt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:54.854870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.854775    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.854870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.854848    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.854920    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.854984    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqlq\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855091 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855051    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855113    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855189    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855286    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.855382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855314    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855703    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855728    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855749    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855763    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855776    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855790    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855809    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855825    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.856625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.855852    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcbt9\" (UniqueName: \"kubernetes.io/projected/d9eb0447-0ceb-4ec2-b9e9-b0d094885335-kube-api-access-gcbt9\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:54.956625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956584    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.956789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956657    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqlq\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.956789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956719    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.956789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956772    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.956942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956807    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.956942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956893    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.956942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956919    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.956978    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.957012    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.957233    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.957326    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.957384    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957538 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.957519    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.957538 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.957526    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.959018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.958995    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.959287 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.959267    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.972677 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.972646    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqlq\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:54.972862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:54.972838    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token\") pod \"e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") " pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:55.032427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.032384    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:55.062896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.062858    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:55.088955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.088915    6389 generic.go:356] "Generic (PLEG): container finished" podID="63e92526-b99e-4623-994c-56d089419dd6" containerID="187a52602b29a5c00425810c307c42111c800ed727716040bb4cf0eb9420abe4" exitCode=0
Oct 30 20:22:55.089082 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.089052    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" event={"ID":"63e92526-b99e-4623-994c-56d089419dd6","Type":"ContainerDied","Data":"187a52602b29a5c00425810c307c42111c800ed727716040bb4cf0eb9420abe4"}
Oct 30 20:22:55.091028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.090996    6389 generic.go:356] "Generic (PLEG): container finished" podID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" containerID="98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693" exitCode=0
Oct 30 20:22:55.091167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.091153    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" event={"ID":"d9eb0447-0ceb-4ec2-b9e9-b0d094885335","Type":"ContainerDied","Data":"98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693"}
Oct 30 20:22:55.091248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.091200    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5" event={"ID":"d9eb0447-0ceb-4ec2-b9e9-b0d094885335","Type":"ContainerDied","Data":"091c8df246147cb60c85bbf0c408e5a618982457db7b448c44727699e7d19171"}
Oct 30 20:22:55.091248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.091229    6389 scope.go:122] "RemoveContainer" containerID="98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693"
Oct 30 20:22:55.091456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.091419    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"
Oct 30 20:22:55.117501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.116264    6389 generic.go:356] "Generic (PLEG): container finished" podID="0c2e82a7-b61b-452e-b956-a9173e4eef49" containerID="ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d" exitCode=0
Oct 30 20:22:55.117501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.116433    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" event={"ID":"0c2e82a7-b61b-452e-b956-a9173e4eef49","Type":"ContainerDied","Data":"ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d"}
Oct 30 20:22:55.117501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.116478    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv" event={"ID":"0c2e82a7-b61b-452e-b956-a9173e4eef49","Type":"ContainerDied","Data":"d8d5c642709562b1994496ae067b716346de261c1d769c10712040d3abe356e3"}
Oct 30 20:22:55.117501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.116557    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"
Oct 30 20:22:55.176151 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.176130    6389 scope.go:122] "RemoveContainer" containerID="98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693"
Oct 30 20:22:55.176508 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:55.176481    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693\": container with ID starting with 98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693 not found: ID does not exist" containerID="98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693"
Oct 30 20:22:55.176595 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.176524    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693"} err="failed to get container status \"98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693\": rpc error: code = NotFound desc = could not find container \"98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693\": container with ID starting with 98915bcd69434545d23d14f02c6ce58778390398ba3c6a9c39d351da0951c693 not found: ID does not exist"
Oct 30 20:22:55.176595 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.176549    6389 scope.go:122] "RemoveContainer" containerID="ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d"
Oct 30 20:22:55.185577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.185439    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"]
Oct 30 20:22:55.190259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.190235    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-k4vv5"]
Oct 30 20:22:55.193600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.193578    6389 scope.go:122] "RemoveContainer" containerID="ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d"
Oct 30 20:22:55.194220 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:55.193812    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d\": container with ID starting with ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d not found: ID does not exist" containerID="ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d"
Oct 30 20:22:55.194220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.193860    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d"} err="failed to get container status \"ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d\": rpc error: code = NotFound desc = could not find container \"ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d\": container with ID starting with ee77e563ffe1d9f79453b6f58b59008bd606b7c3a651a0d33d1b436b6001913d not found: ID does not exist"
Oct 30 20:22:55.202937 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.202912    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"]
Oct 30 20:22:55.208217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.208192    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-b8qfv"]
Oct 30 20:22:55.214961 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.214942    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:22:55.345523 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.345494    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"]
Oct 30 20:22:55.345908 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:22:55.345562    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6250214c_7743_4a40_bac4_152a3349becd.slice/crio-d1532d8aeb19dacd227d43ce3ad91c5203fcf6d45dc2c97bcdfcad0df9737f50 WatchSource:0}: Error finding container d1532d8aeb19dacd227d43ce3ad91c5203fcf6d45dc2c97bcdfcad0df9737f50: Status 404 returned error can't find the container with id d1532d8aeb19dacd227d43ce3ad91c5203fcf6d45dc2c97bcdfcad0df9737f50
Oct 30 20:22:55.348235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.348206    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:55.348328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.348264    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:55.348328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.348302    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:22:55.360993 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.360965    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361079 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361040    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361101    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361175    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361221    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361282    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361361    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361490    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.361574 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361556    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs\") pod \"63e92526-b99e-4623-994c-56d089419dd6\" (UID: \"63e92526-b99e-4623-994c-56d089419dd6\") "
Oct 30 20:22:55.362957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.361357    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:55.363096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.362289    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:55.363250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.362413    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:55.363375 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.362496    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:55.363487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.362937    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:55.364955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.364933    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:55.368014 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.367459    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "kube-api-access-x2p9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:55.368014 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.367494    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:55.370334 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.370300    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo" pod "63e92526-b99e-4623-994c-56d089419dd6" (UID: "63e92526-b99e-4623-994c-56d089419dd6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:55.463262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463233    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2p9t\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-kube-api-access-x2p9t\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463260    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463271    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e92526-b99e-4623-994c-56d089419dd6-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463279    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463287    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63e92526-b99e-4623-994c-56d089419dd6-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463295    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463305    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463317    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63e92526-b99e-4623-994c-56d089419dd6-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.463503 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.463325    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63e92526-b99e-4623-994c-56d089419dd6-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:55.794190 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:55.794095    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:22:55.915430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.915399    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c2e82a7-b61b-452e-b956-a9173e4eef49" path="/var/lib/kubelet/pods/0c2e82a7-b61b-452e-b956-a9173e4eef49/volumes"
Oct 30 20:22:55.915912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:55.915898    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9eb0447-0ceb-4ec2-b9e9-b0d094885335" path="/var/lib/kubelet/pods/d9eb0447-0ceb-4ec2-b9e9-b0d094885335/volumes"
Oct 30 20:22:56.051900 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.051762    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:56.156826 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.156780    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" event={"ID":"6250214c-7743-4a40-bac4-152a3349becd","Type":"ContainerStarted","Data":"4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31"}
Oct 30 20:22:56.157050 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.156862    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" event={"ID":"6250214c-7743-4a40-bac4-152a3349becd","Type":"ContainerStarted","Data":"d1532d8aeb19dacd227d43ce3ad91c5203fcf6d45dc2c97bcdfcad0df9737f50"}
Oct 30 20:22:56.159195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.158649    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"
Oct 30 20:22:56.159195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.158662    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm" event={"ID":"63e92526-b99e-4623-994c-56d089419dd6","Type":"ContainerDied","Data":"8171a58af4decc40ebd779f8cd346f4976f7483bed1d93e56cdf7cd1165e849e"}
Oct 30 20:22:56.159195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.158725    6389 scope.go:122] "RemoveContainer" containerID="187a52602b29a5c00425810c307c42111c800ed727716040bb4cf0eb9420abe4"
Oct 30 20:22:56.202489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.202438    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podStartSLOduration=2.202423428 podStartE2EDuration="2.202423428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:22:56.200910615 +0000 UTC m=+1096.400483543" watchObservedRunningTime="2026-10-30 20:22:56.202423428 +0000 UTC m=+1096.401996356"
Oct 30 20:22:56.206931 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.206906    6389 patch_prober.go:29] interesting pod/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:22:56.207051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.206957    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:22:56.225138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.221251    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"]
Oct 30 20:22:56.255020 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.254981    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-44135b-openshift-default-76f454bf89-fw2hm"]
Oct 30 20:22:56.278998 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.278963    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:56.935153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:56.929581    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:57.043142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.040442    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:22:57.045840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.045803    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 20:22:57.045972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.045870    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 20:22:57.673048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.672959    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:57.843694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.843661    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:57.958139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.941772    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63e92526-b99e-4623-994c-56d089419dd6" path="/var/lib/kubelet/pods/63e92526-b99e-4623-994c-56d089419dd6/volumes"
Oct 30 20:22:57.981787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.981754    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:57.987096 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.987074    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"]
Oct 30 20:22:57.993405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.993380    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-909519-pdn8x"]
Oct 30 20:22:57.995747 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:57.995521    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:22:58.054460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.054425    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 20:22:58.054688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.054660    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 20:22:58.089494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.089443    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:58.151145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.136314    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:22:58.166618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166594    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.166735 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166627    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.166735 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166662    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.166735 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166695    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.166862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166736    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.166862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166764    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.166862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166794    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.166862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166838    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.166990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166889    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.166990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166939    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.166990 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.166971    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.167082 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167059    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.167236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167108    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.167295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167273    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.167332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167309    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.167411 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167395    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket\") pod \"fbad6255-cd3b-49c6-bf1f-982304f50150\" (UID: \"fbad6255-cd3b-49c6-bf1f-982304f50150\") "
Oct 30 20:22:58.167506 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167489    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.167591 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167576    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r\") pod \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\" (UID: \"e524ab0d-f1de-4473-bf97-30b7268c9a6c\") "
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.167668    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.168437    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.168462    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.168644    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.168658    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.168956    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.169084    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.169102    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.169425    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:58.169850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.169488    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "kube-api-access-tmghc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:58.170743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.170691    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:58.170872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.170845    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:58.171529 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.171497    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:58.172240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.172206    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy" pod "fbad6255-cd3b-49c6-bf1f-982304f50150" (UID: "fbad6255-cd3b-49c6-bf1f-982304f50150"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.172372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.172332    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.172632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.172579    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.172632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.172617    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:58.172806 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.172689    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.173997 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.173962    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:22:58.174466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.174441    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r" pod "e524ab0d-f1de-4473-bf97-30b7268c9a6c" (UID: "e524ab0d-f1de-4473-bf97-30b7268c9a6c"). InnerVolumeSpecName "kube-api-access-rm22r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:58.181407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.180430    6389 generic.go:356] "Generic (PLEG): container finished" podID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerID="11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312" exitCode=0
Oct 30 20:22:58.181407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.180617    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" event={"ID":"fbad6255-cd3b-49c6-bf1f-982304f50150","Type":"ContainerDied","Data":"11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312"}
Oct 30 20:22:58.181407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.180670    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft" event={"ID":"fbad6255-cd3b-49c6-bf1f-982304f50150","Type":"ContainerDied","Data":"df6dc70dc5d6ff7ee2a62473db3bd8de8ae56204b34a78b57eaf691b9404dece"}
Oct 30 20:22:58.181407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.180699    6389 scope.go:122] "RemoveContainer" containerID="11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312"
Oct 30 20:22:58.181407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.180850    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.188922    6389 generic.go:356] "Generic (PLEG): container finished" podID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerID="60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5" exitCode=0
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.189074    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" event={"ID":"af4ccf0b-c95d-426c-8670-1cce9d9c9699","Type":"ContainerDied","Data":"60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5"}
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.189250    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z" event={"ID":"af4ccf0b-c95d-426c-8670-1cce9d9c9699","Type":"ContainerDied","Data":"2f37bcf891123d8f5bfbb29a0cc1dc6c60213997f98aed8a9ab6c76b08b5146a"}
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.189365    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.191595    6389 generic.go:356] "Generic (PLEG): container finished" podID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerID="f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49" exitCode=0
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.192144    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.193794    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" event={"ID":"e524ab0d-f1de-4473-bf97-30b7268c9a6c","Type":"ContainerDied","Data":"f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49"}
Oct 30 20:22:58.194499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.193847    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t" event={"ID":"e524ab0d-f1de-4473-bf97-30b7268c9a6c","Type":"ContainerDied","Data":"54ce3a859d5dddf35db9c00c24164a549c2e25354a5d3b38167699258cd9db7b"}
Oct 30 20:22:58.209196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.209171    6389 scope.go:122] "RemoveContainer" containerID="11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312"
Oct 30 20:22:58.211834 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:58.211639    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312\": container with ID starting with 11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312 not found: ID does not exist" containerID="11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312"
Oct 30 20:22:58.211834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.211682    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312"} err="failed to get container status \"11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312\": rpc error: code = NotFound desc = could not find container \"11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312\": container with ID starting with 11a87bd1b54eebeda3b1d4dfe294e996fce5356fe064721b5545ddda43eab312 not found: ID does not exist"
Oct 30 20:22:58.211834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.211708    6389 scope.go:122] "RemoveContainer" containerID="60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5"
Oct 30 20:22:58.261178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.259462    6389 scope.go:122] "RemoveContainer" containerID="60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5"
Oct 30 20:22:58.261178 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:58.259783    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5\": container with ID starting with 60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5 not found: ID does not exist" containerID="60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5"
Oct 30 20:22:58.261178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.259810    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5"} err="failed to get container status \"60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5\": rpc error: code = NotFound desc = could not find container \"60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5\": container with ID starting with 60359e22fde031d2b404415990db066c5d4959a550d98680900817f2da7acbb5 not found: ID does not exist"
Oct 30 20:22:58.261178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.259831    6389 scope.go:122] "RemoveContainer" containerID="f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49"
Oct 30 20:22:58.262576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.262553    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"]
Oct 30 20:22:58.267765 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.267741    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-7msft"]
Oct 30 20:22:58.270804 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.269488    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.270804 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.269566    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.270804 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.270747    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.272458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.272269    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.272458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.272323    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.272458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.272377    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.272458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.272426    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.272718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.272468    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.272718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.272535    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket\") pod \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\" (UID: \"af4ccf0b-c95d-426c-8670-1cce9d9c9699\") "
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273183    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273208    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273223    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273237    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbad6255-cd3b-49c6-bf1f-982304f50150-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273250    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273264    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273277    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273292    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273306    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273332    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm22r\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-kube-api-access-rm22r\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273350    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmghc\" (UniqueName: \"kubernetes.io/projected/fbad6255-cd3b-49c6-bf1f-982304f50150-kube-api-access-tmghc\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273364    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273378    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273393    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbad6255-cd3b-49c6-bf1f-982304f50150-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273406    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbad6255-cd3b-49c6-bf1f-982304f50150-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273420    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.273975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.273429    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e524ab0d-f1de-4473-bf97-30b7268c9a6c-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.277100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.277086    6389 scope.go:122] "RemoveContainer" containerID="f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49"
Oct 30 20:22:58.277421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.277401    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "kube-api-access-ps429". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.277685    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:22:58.277780    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49\": container with ID starting with f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49 not found: ID does not exist" containerID="f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49"
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.277817    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49"} err="failed to get container status \"f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49\": rpc error: code = NotFound desc = could not find container \"f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49\": container with ID starting with f5f440ad54d4a947bb3751bc10bf0606ecb41162fe1b9ca7050099bda3393c49 not found: ID does not exist"
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.278022    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.278083    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.278168    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.279172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.278410    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.279660 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.279638    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:58.280888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.280242    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"]
Oct 30 20:22:58.281301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.281277    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:58.281455 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.281433    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy" pod "af4ccf0b-c95d-426c-8670-1cce9d9c9699" (UID: "af4ccf0b-c95d-426c-8670-1cce9d9c9699"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:58.283028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.283003    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:58.284238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.284215    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-9qk8t"]
Oct 30 20:22:58.374385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374301    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374334    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374347    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374358    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374372    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374384    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps429\" (UniqueName: \"kubernetes.io/projected/af4ccf0b-c95d-426c-8670-1cce9d9c9699-kube-api-access-ps429\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374693 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374394    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374693 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374403    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ccf0b-c95d-426c-8670-1cce9d9c9699-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.374693 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.374411    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ccf0b-c95d-426c-8670-1cce9d9c9699-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:58.450051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.449750    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:58.520869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.520838    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"]
Oct 30 20:22:58.551565 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:58.551533    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-75e734-openshift-default-6d7bc99d4c-rpb5z"]
Oct 30 20:22:59.043147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.042529    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 20:22:59.043147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.042592    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 20:22:59.200728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.200692    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:22:59.657044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.655288    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690362    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690417    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690529    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690570    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690601    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690691    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690765    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690821    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.690908    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert\") pod \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\" (UID: \"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf\") "
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.691730    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.692008    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.692217    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.694987    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.695578    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.697254    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.697320    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.697518    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "kube-api-access-khszw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:59.701155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.698591    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token" pod "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" (UID: "074ea9fc-abf6-42a0-bd1c-3f472d66dfaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:59.755325 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.755295    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792086    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792147    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792169    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792208    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792234    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792296    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792317    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792360    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792427    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket\") pod \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\" (UID: \"d7bfa1e7-769b-4f38-bc5c-677f603542c0\") "
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792711    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792723    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792732    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792742    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khszw\" (UniqueName: \"kubernetes.io/projected/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-kube-api-access-khszw\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792752    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792761    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792770    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792778    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792787    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.793245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.792985    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.794572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.794544    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.795549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.795199    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.795549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.795279    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.795848 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.795807    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:59.797312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.797267    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:59.797936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.797909    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "kube-api-access-r64xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:59.801382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.801358    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:59.801499 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.801481    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo" pod "d7bfa1e7-769b-4f38-bc5c-677f603542c0" (UID: "d7bfa1e7-769b-4f38-bc5c-677f603542c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:59.894205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894168    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894198    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894208    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894218    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894473 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894227    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894473 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894236    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894473 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894244    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r64xv\" (UniqueName: \"kubernetes.io/projected/d7bfa1e7-769b-4f38-bc5c-677f603542c0-kube-api-access-r64xv\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894473 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894254    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.894473 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.894262    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bfa1e7-769b-4f38-bc5c-677f603542c0-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:59.914417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.914352    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" path="/var/lib/kubelet/pods/af4ccf0b-c95d-426c-8670-1cce9d9c9699/volumes"
Oct 30 20:22:59.914787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.914774    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d26faff4-0ee8-4790-ada6-d93808fe7711" path="/var/lib/kubelet/pods/d26faff4-0ee8-4790-ada6-d93808fe7711/volumes"
Oct 30 20:22:59.915204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.915191    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" path="/var/lib/kubelet/pods/e524ab0d-f1de-4473-bf97-30b7268c9a6c/volumes"
Oct 30 20:22:59.915908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:22:59.915894    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" path="/var/lib/kubelet/pods/fbad6255-cd3b-49c6-bf1f-982304f50150/volumes"
Oct 30 20:23:00.040142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.038955    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 20:23:00.040142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.039025    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 20:23:00.050933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.050903    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:00.212887 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.212803    6389 generic.go:356] "Generic (PLEG): container finished" podID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerID="3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d" exitCode=0
Oct 30 20:23:00.212887 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.212851    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"
Oct 30 20:23:00.213058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.212885    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" event={"ID":"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf","Type":"ContainerDied","Data":"3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d"}
Oct 30 20:23:00.213058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.212948    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78" event={"ID":"074ea9fc-abf6-42a0-bd1c-3f472d66dfaf","Type":"ContainerDied","Data":"181124f5e2337c32c5dd724096372d35c62bcf09d5efaebf9e3fe0a87151a24f"}
Oct 30 20:23:00.213058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.212972    6389 scope.go:122] "RemoveContainer" containerID="3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d"
Oct 30 20:23:00.214510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.214482    6389 generic.go:356] "Generic (PLEG): container finished" podID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerID="5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1" exitCode=0
Oct 30 20:23:00.214638 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.214521    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"
Oct 30 20:23:00.214638 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.214570    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" event={"ID":"d7bfa1e7-769b-4f38-bc5c-677f603542c0","Type":"ContainerDied","Data":"5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1"}
Oct 30 20:23:00.214638 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.214629    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s" event={"ID":"d7bfa1e7-769b-4f38-bc5c-677f603542c0","Type":"ContainerDied","Data":"a13d8eb2a5e772dad4bdeedc915c27197055004551fd3e4e99c81af3b9736da0"}
Oct 30 20:23:00.247285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.247259    6389 scope.go:122] "RemoveContainer" containerID="3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d"
Oct 30 20:23:00.247687 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:00.247666    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d\": container with ID starting with 3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d not found: ID does not exist" containerID="3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d"
Oct 30 20:23:00.247791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.247698    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d"} err="failed to get container status \"3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d\": rpc error: code = NotFound desc = could not find container \"3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d\": container with ID starting with 3887298758ba1082559f727968e68e778d7bb1187174146bf791ef6de193448d not found: ID does not exist"
Oct 30 20:23:00.247791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.247719    6389 scope.go:122] "RemoveContainer" containerID="5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1"
Oct 30 20:23:00.258707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.258691    6389 scope.go:122] "RemoveContainer" containerID="5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1"
Oct 30 20:23:00.259052 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:00.259031    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1\": container with ID starting with 5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1 not found: ID does not exist" containerID="5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1"
Oct 30 20:23:00.259141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.259057    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1"} err="failed to get container status \"5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1\": rpc error: code = NotFound desc = could not find container \"5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1\": container with ID starting with 5a4190f69a034cc2153867f3f7a06c3154f0b9babd565bf749427ea9ffc785d1 not found: ID does not exist"
Oct 30 20:23:00.272078 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.272056    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"]
Oct 30 20:23:00.281340 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.281315    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-wmt78"]
Oct 30 20:23:00.306755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.306718    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"]
Oct 30 20:23:00.349027 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.348994    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-9c901b-openshift-default-bc4d55965-dcp8s"]
Oct 30 20:23:00.888039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:00.888008    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:01.043141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:01.039886    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 20:23:01.043141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:01.039948    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 20:23:01.046143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:01.045525    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:01.872033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:01.871997    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:01.929234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:01.929191    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" path="/var/lib/kubelet/pods/074ea9fc-abf6-42a0-bd1c-3f472d66dfaf/volumes"
Oct 30 20:23:01.929863 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:01.929831    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" path="/var/lib/kubelet/pods/d7bfa1e7-769b-4f38-bc5c-677f603542c0/volumes"
Oct 30 20:23:02.033839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.033796    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 20:23:02.034016 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.033848    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 20:23:02.706060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.706019    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:02.772011 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.771759    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:23:02.834448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.834415    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume\") pod \"a9d4cbc9-d47a-45fb-b927-115458a1785b\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") "
Oct 30 20:23:02.834662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.834646    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6\") pod \"a9d4cbc9-d47a-45fb-b927-115458a1785b\" (UID: \"a9d4cbc9-d47a-45fb-b927-115458a1785b\") "
Oct 30 20:23:02.834806 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.834783    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume" pod "a9d4cbc9-d47a-45fb-b927-115458a1785b" (UID: "a9d4cbc9-d47a-45fb-b927-115458a1785b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:02.835019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.834998    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d4cbc9-d47a-45fb-b927-115458a1785b-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:02.837131 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.837090    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6" pod "a9d4cbc9-d47a-45fb-b927-115458a1785b" (UID: "a9d4cbc9-d47a-45fb-b927-115458a1785b"). InnerVolumeSpecName "kube-api-access-58xw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:02.935775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:02.935745    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/a9d4cbc9-d47a-45fb-b927-115458a1785b-kube-api-access-58xw6\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:03.037966 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.037892    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:23:03.038363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.038334    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:23:03.038953 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.038933    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:23:03.277713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.277676    6389 generic.go:356] "Generic (PLEG): container finished" podID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerID="f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb" exitCode=0
Oct 30 20:23:03.278012 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.277732    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"
Oct 30 20:23:03.278151 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.277760    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" event={"ID":"a9d4cbc9-d47a-45fb-b927-115458a1785b","Type":"ContainerDied","Data":"f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb"}
Oct 30 20:23:03.278220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.278148    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd" event={"ID":"a9d4cbc9-d47a-45fb-b927-115458a1785b","Type":"ContainerDied","Data":"c2f5ed0e15c569488c539356d92a033f3261f68a52b89d9acf019d96424632ba"}
Oct 30 20:23:03.278220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.278189    6389 scope.go:122] "RemoveContainer" containerID="f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb"
Oct 30 20:23:03.293376 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.293353    6389 scope.go:122] "RemoveContainer" containerID="f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb"
Oct 30 20:23:03.293760 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:03.293740    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb\": container with ID starting with f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb not found: ID does not exist" containerID="f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb"
Oct 30 20:23:03.293830 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.293769    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb"} err="failed to get container status \"f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb\": rpc error: code = NotFound desc = could not find container \"f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb\": container with ID starting with f57a4d226d588043b2b7a24bb24414c15d14e3eff484e6c019c64d03d79500cb not found: ID does not exist"
Oct 30 20:23:03.310967 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.310940    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"]
Oct 30 20:23:03.315740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.315718    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-909519-7b9d6f8dc-7c2qd"]
Oct 30 20:23:03.530515 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.530480    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:03.914189 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:03.914160    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" path="/var/lib/kubelet/pods/a9d4cbc9-d47a-45fb-b927-115458a1785b/volumes"
Oct 30 20:23:04.500945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:04.500916    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:05.343263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:05.343226    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:06.282295 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:06.282266    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.028273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.028232    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.246458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.244474    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.382946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.382767    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.492152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.492088    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.573246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.573195    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"]
Oct 30 20:23:07.585417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.585351    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy" containerID="cri-o://b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178" gracePeriod=30
Oct 30 20:23:07.587583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.587545    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"]
Oct 30 20:23:07.587790 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.587766    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy" containerID="cri-o://78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6" gracePeriod=30
Oct 30 20:23:07.595419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.595391    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"]
Oct 30 20:23:07.595629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.595605    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy" containerID="cri-o://d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7" gracePeriod=30
Oct 30 20:23:07.637230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.636312    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.756443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.756389    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:07.950362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:07.950277    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:08.257983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:08.257942    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:09.164428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:09.164390    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:10.162465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:10.162424    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:10.999682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:10.999636    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:11.252239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:11.252151    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:11.976261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:11.976225    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:12.730557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.724996    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"]
Oct 30 20:23:12.734722 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.734691    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"]
Oct 30 20:23:12.751160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.751089    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"]
Oct 30 20:23:12.829315 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.829278    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:12.938312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.938287    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:23:12.949311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.949259    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:12.965036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.962881    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:23:12.987704 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:12.987673    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:23:13.049507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049468    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.049697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049525    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.049697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049558    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.049697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049609    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.049697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049645    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.049697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049680    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.050002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049725    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049823    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.050002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049903    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.050002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049931    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.049972    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050059    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.050309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050088    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050171    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data\") pod \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\" (UID: \"48a40eda-7234-4bdc-b5f5-0509ad61aaa9\") "
Oct 30 20:23:13.050309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050209    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050240    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050309 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050277    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.050648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.050320    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo\") pod \"b79139b8-03e9-48f2-b257-3177cb352477\" (UID: \"b79139b8-03e9-48f2-b257-3177cb352477\") "
Oct 30 20:23:13.052461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.051640    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:13.052461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.051730    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.052461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.051972    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.052461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.052187    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.053491 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.053430    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.054855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.054160    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:13.054855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.054435    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.054855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.054618    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.055585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.055401    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.055585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.055422    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.055585 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.055476    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:13.055882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.055848    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.055882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.055865    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:13.056207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.056169    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "kube-api-access-8c6zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:13.056527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.056502    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:13.056733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.056710    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy" pod "b79139b8-03e9-48f2-b257-3177cb352477" (UID: "b79139b8-03e9-48f2-b257-3177cb352477"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.056956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.056934    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:13.058273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.058252    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25" pod "48a40eda-7234-4bdc-b5f5-0509ad61aaa9" (UID: "48a40eda-7234-4bdc-b5f5-0509ad61aaa9"). InnerVolumeSpecName "kube-api-access-njr25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:13.151717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.151634    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.151880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.151773    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.151880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.151828    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.151880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.151867    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.152057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.151917    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.152057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.151980    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.152057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152011    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.152239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152075    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.152239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152110    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl\") pod \"c463c1ef-cffb-492d-94cc-61e42e93e99d\" (UID: \"c463c1ef-cffb-492d-94cc-61e42e93e99d\") "
Oct 30 20:23:13.152596 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152568    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152596    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152611    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152625    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152638    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152651    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152666    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152680    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.152700 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152693    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152708    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c6zl\" (UniqueName: \"kubernetes.io/projected/b79139b8-03e9-48f2-b257-3177cb352477-kube-api-access-8c6zl\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152722    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152737    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79139b8-03e9-48f2-b257-3177cb352477-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152752    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79139b8-03e9-48f2-b257-3177cb352477-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152765    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152779    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152793    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152807    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njr25\" (UniqueName: \"kubernetes.io/projected/48a40eda-7234-4bdc-b5f5-0509ad61aaa9-kube-api-access-njr25\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152821    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79139b8-03e9-48f2-b257-3177cb352477-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.153159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.152836    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.153688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.153224    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:13.153688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.153283    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.153688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.153337    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.153688 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.153381    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.154810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.154783    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:13.155543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.155516    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "kube-api-access-58ghl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:13.155739 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.155715    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:13.155871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.155847    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo" pod "c463c1ef-cffb-492d-94cc-61e42e93e99d" (UID: "c463c1ef-cffb-492d-94cc-61e42e93e99d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:13.254299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254260    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254299 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254301    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58ghl\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-kube-api-access-58ghl\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254317    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254332    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254347    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254361    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254375    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254389    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c463c1ef-cffb-492d-94cc-61e42e93e99d-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.254526 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.254404    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c463c1ef-cffb-492d-94cc-61e42e93e99d-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:13.354987 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.354950    6389 generic.go:356] "Generic (PLEG): container finished" podID="b79139b8-03e9-48f2-b257-3177cb352477" containerID="78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6" exitCode=0
Oct 30 20:23:13.355198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.355008    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"
Oct 30 20:23:13.355198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.355045    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" event={"ID":"b79139b8-03e9-48f2-b257-3177cb352477","Type":"ContainerDied","Data":"78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6"}
Oct 30 20:23:13.355198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.355094    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v" event={"ID":"b79139b8-03e9-48f2-b257-3177cb352477","Type":"ContainerDied","Data":"236c50c6db02fb2a6dd18ac300c0ea1734f74c019ce0bb9c02077dac6db7e85e"}
Oct 30 20:23:13.355198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.355140    6389 scope.go:122] "RemoveContainer" containerID="78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6"
Oct 30 20:23:13.356504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.356472    6389 generic.go:356] "Generic (PLEG): container finished" podID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerID="b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178" exitCode=0
Oct 30 20:23:13.356607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.356517    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"
Oct 30 20:23:13.356899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.356562    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" event={"ID":"48a40eda-7234-4bdc-b5f5-0509ad61aaa9","Type":"ContainerDied","Data":"b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178"}
Oct 30 20:23:13.356970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.356936    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7" event={"ID":"48a40eda-7234-4bdc-b5f5-0509ad61aaa9","Type":"ContainerDied","Data":"76dc9afe6885f16da293e8dcda84305fd9e982064f99b04cd8a4c3cca1bc786f"}
Oct 30 20:23:13.358268 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.358234    6389 generic.go:356] "Generic (PLEG): container finished" podID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerID="d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7" exitCode=0
Oct 30 20:23:13.358388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.358279    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" event={"ID":"c463c1ef-cffb-492d-94cc-61e42e93e99d","Type":"ContainerDied","Data":"d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7"}
Oct 30 20:23:13.358388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.358289    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"
Oct 30 20:23:13.358388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.358318    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt" event={"ID":"c463c1ef-cffb-492d-94cc-61e42e93e99d","Type":"ContainerDied","Data":"2be3827687f8226fd2d58ae273f785d292086a1c25cef452f83bda708e353011"}
Oct 30 20:23:13.368360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.368257    6389 scope.go:122] "RemoveContainer" containerID="78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6"
Oct 30 20:23:13.368760 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:13.368695    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6\": container with ID starting with 78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6 not found: ID does not exist" containerID="78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6"
Oct 30 20:23:13.368870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.368771    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6"} err="failed to get container status \"78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6\": rpc error: code = NotFound desc = could not find container \"78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6\": container with ID starting with 78191004d1e04d30c9f53de0a00918c293120f2cd2ae3237b694de66af9b69a6 not found: ID does not exist"
Oct 30 20:23:13.368870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.368797    6389 scope.go:122] "RemoveContainer" containerID="b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178"
Oct 30 20:23:13.381586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.381569    6389 scope.go:122] "RemoveContainer" containerID="b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178"
Oct 30 20:23:13.381867 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:13.381850    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178\": container with ID starting with b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178 not found: ID does not exist" containerID="b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178"
Oct 30 20:23:13.381938 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.381875    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178"} err="failed to get container status \"b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178\": rpc error: code = NotFound desc = could not find container \"b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178\": container with ID starting with b6b423b39fdda70492ef3e690a13b628a473c8b8eba64a764201bc4439353178 not found: ID does not exist"
Oct 30 20:23:13.381938 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.381891    6389 scope.go:122] "RemoveContainer" containerID="d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7"
Oct 30 20:23:13.391356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.391335    6389 scope.go:122] "RemoveContainer" containerID="d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7"
Oct 30 20:23:13.391560 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.391538    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"]
Oct 30 20:23:13.391653 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:13.391635    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7\": container with ID starting with d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7 not found: ID does not exist" containerID="d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7"
Oct 30 20:23:13.391698 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.391661    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7"} err="failed to get container status \"d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7\": rpc error: code = NotFound desc = could not find container \"d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7\": container with ID starting with d3a27b4b637119031fd96adfda0787eff632468e7a4e8afec6783c582f0c08d7 not found: ID does not exist"
Oct 30 20:23:13.398149 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.398104    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-lgx5v"]
Oct 30 20:23:13.411147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.411089    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"]
Oct 30 20:23:13.417889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.417866    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-gnjgt"]
Oct 30 20:23:13.430540 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.430521    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"]
Oct 30 20:23:13.437220 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.437195    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-909519-openshift-default-665577cd95-kwgk7"]
Oct 30 20:23:13.784350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.784316    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:13.913919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.913887    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" path="/var/lib/kubelet/pods/48a40eda-7234-4bdc-b5f5-0509ad61aaa9/volumes"
Oct 30 20:23:13.914361 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.914348    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b79139b8-03e9-48f2-b257-3177cb352477" path="/var/lib/kubelet/pods/b79139b8-03e9-48f2-b257-3177cb352477/volumes"
Oct 30 20:23:13.914899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:13.914876    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" path="/var/lib/kubelet/pods/c463c1ef-cffb-492d-94cc-61e42e93e99d/volumes"
Oct 30 20:23:14.560285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:14.560245    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:15.472727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:15.472697    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:15.890818 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:15.890738    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:16.317480 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:16.317446    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:17.242004 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:17.241969    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:17.995333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:17.995298    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:18.127549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.127507    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"]
Oct 30 20:23:18.200041 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.200009    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:18.304229 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.304144    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:18.597579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.597496    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:18.689527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.689492    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"]
Oct 30 20:23:18.690085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690061    6389 cpu_manager.go:414] "containerMap: removing container" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy"
Oct 30 20:23:18.690085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690084    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy"
Oct 30 20:23:18.690085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690097    6389 cpu_manager.go:414] "containerMap: removing container" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerName="payload-pre-processing"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690105    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerName="payload-pre-processing"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690152    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690160    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690178    6389 cpu_manager.go:414] "containerMap: removing container" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690186    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690197    6389 cpu_manager.go:414] "containerMap: removing container" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690204    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690218    6389 cpu_manager.go:414] "containerMap: removing container" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690223    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690232    6389 cpu_manager.go:414] "containerMap: removing container" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690236    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690244    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690248    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690260    6389 cpu_manager.go:414] "containerMap: removing container" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690267    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690275    6389 cpu_manager.go:414] "containerMap: removing container" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690281    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690378    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="63e92526-b99e-4623-994c-56d089419dd6" containerName="istio-proxy"
Oct 30 20:23:18.690374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690386    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="af4ccf0b-c95d-426c-8670-1cce9d9c9699" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690392    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9d4cbc9-d47a-45fb-b927-115458a1785b" containerName="payload-pre-processing"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690397    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="074ea9fc-abf6-42a0-bd1c-3f472d66dfaf" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690403    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="c463c1ef-cffb-492d-94cc-61e42e93e99d" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690410    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="b79139b8-03e9-48f2-b257-3177cb352477" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690416    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7bfa1e7-769b-4f38-bc5c-677f603542c0" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690426    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="e524ab0d-f1de-4473-bf97-30b7268c9a6c" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690434    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="48a40eda-7234-4bdc-b5f5-0509ad61aaa9" containerName="istio-proxy"
Oct 30 20:23:18.691298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.690442    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbad6255-cd3b-49c6-bf1f-982304f50150" containerName="istio-proxy"
Oct 30 20:23:18.694200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.694178    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.704734 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.703475    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"]
Oct 30 20:23:18.735630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.735590    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:23:18.735793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.735661    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:23:18.811726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.811690    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9x7\" (UniqueName: \"kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.811726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.811730    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.811920 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.811864    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.912668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.912578    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9x7\" (UniqueName: \"kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.912668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.912626    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.912887 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.912697    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.913372 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.913346    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.915170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.915150    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:18.931248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:18.931222    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9x7\" (UniqueName: \"kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7\") pod \"maas-api-revoke-sub-609ba8d0-2z7lm\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:19.011079 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.011051    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:19.034678 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.034587    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:19.309012 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.297235    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"]
Oct 30 20:23:19.324764 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.323530    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:19.415921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.415889    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm" event={"ID":"7463ae95-2cbb-4615-abd0-eba3df86c4b1","Type":"ContainerStarted","Data":"5a209bf3c977b478827590ddde0095e4cca363f4bbe709d6a08efa80147d91e4"}
Oct 30 20:23:19.429084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.429054    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:19.974011 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:19.973979    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:20.423910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:20.423879    6389 generic.go:356] "Generic (PLEG): container finished" podID="7463ae95-2cbb-4615-abd0-eba3df86c4b1" containerID="53bc87a2e2e55397f96ac993e683e52ca62b25f9653c64d9fd3ffe1321a6c8cb" exitCode=0
Oct 30 20:23:20.423910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:20.423915    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm" event={"ID":"7463ae95-2cbb-4615-abd0-eba3df86c4b1","Type":"ContainerDied","Data":"53bc87a2e2e55397f96ac993e683e52ca62b25f9653c64d9fd3ffe1321a6c8cb"}
Oct 30 20:23:20.859579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:20.859544    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:21.572673 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.572649    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:21.678626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.678588    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:21.744607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.744572    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca\") pod \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") "
Oct 30 20:23:21.744800 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.744616    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7\" (UniqueName: \"kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7\") pod \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") "
Oct 30 20:23:21.744800 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.744724    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca\") pod \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\" (UID: \"7463ae95-2cbb-4615-abd0-eba3df86c4b1\") "
Oct 30 20:23:21.744970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.744946    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca" pod "7463ae95-2cbb-4615-abd0-eba3df86c4b1" (UID: "7463ae95-2cbb-4615-abd0-eba3df86c4b1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:21.746725 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.746703    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca" pod "7463ae95-2cbb-4615-abd0-eba3df86c4b1" (UID: "7463ae95-2cbb-4615-abd0-eba3df86c4b1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:21.746830 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.746814    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7" pod "7463ae95-2cbb-4615-abd0-eba3df86c4b1" (UID: "7463ae95-2cbb-4615-abd0-eba3df86c4b1"). InnerVolumeSpecName "kube-api-access-9k9x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:21.845795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.845763    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:21.845795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.845789    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7463ae95-2cbb-4615-abd0-eba3df86c4b1-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:21.845795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:21.845800    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k9x7\" (UniqueName: \"kubernetes.io/projected/7463ae95-2cbb-4615-abd0-eba3df86c4b1-kube-api-access-9k9x7\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:22.439095 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:22.439063    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"
Oct 30 20:23:22.439255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:22.439088    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm" event={"ID":"7463ae95-2cbb-4615-abd0-eba3df86c4b1","Type":"ContainerDied","Data":"5a209bf3c977b478827590ddde0095e4cca363f4bbe709d6a08efa80147d91e4"}
Oct 30 20:23:22.439255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:22.439179    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a209bf3c977b478827590ddde0095e4cca363f4bbe709d6a08efa80147d91e4"
Oct 30 20:23:22.455723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:22.455699    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:23.128363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:23.128301    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerName="payload-processing" containerID="cri-o://a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc" gracePeriod=25
Oct 30 20:23:23.382789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:23.382702    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:24.102617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:24.102560    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:24.828918 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:24.828865    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:25.777451 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:25.777395    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:25.792889 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:25.792853    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:23:26.278793 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:26.278763    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:23:26.502995 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:26.502965    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:27.028243 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.028203    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"]
Oct 30 20:23:27.035757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.035726    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9660b20-ppqcz"]
Oct 30 20:23:27.285069 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.284994    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:27.790936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.790896    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main" containerID="cri-o://e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002" gracePeriod=45
Oct 30 20:23:27.803572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.803524    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main" containerID="cri-o://ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab" gracePeriod=45
Oct 30 20:23:27.816893 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.816861    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main" containerID="cri-o://00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa" gracePeriod=45
Oct 30 20:23:27.914387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:27.914359    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2fa1bf6-2042-4ef5-a549-d0066830bb13" path="/var/lib/kubelet/pods/c2fa1bf6-2042-4ef5-a549-d0066830bb13/volumes"
Oct 30 20:23:27.940781 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:27.940748    6389 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42f2aa64_eccc_470b_8500_a0b0a2440c41.slice/crio-ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 20:23:28.094019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.093988    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:23:28.098876 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.098821    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:28.108089 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108065    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5\" (UniqueName: \"kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5\") pod \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") "
Oct 30 20:23:28.108236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108216    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm\") pod \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") "
Oct 30 20:23:28.108329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108255    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home\") pod \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") "
Oct 30 20:23:28.108329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108324    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir\") pod \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") "
Oct 30 20:23:28.108442 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108410    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs\" (UniqueName: \"kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs\") pod \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") "
Oct 30 20:23:28.108495 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108451    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache\") pod \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\" (UID: \"452532ae-6a10-49ca-85dd-c9480c1a0cc8\") "
Oct 30 20:23:28.108813 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108778    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir" pod "452532ae-6a10-49ca-85dd-c9480c1a0cc8" (UID: "452532ae-6a10-49ca-85dd-c9480c1a0cc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.109090 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.108915    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home" pod "452532ae-6a10-49ca-85dd-c9480c1a0cc8" (UID: "452532ae-6a10-49ca-85dd-c9480c1a0cc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.109198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.109165    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache" pod "452532ae-6a10-49ca-85dd-c9480c1a0cc8" (UID: "452532ae-6a10-49ca-85dd-c9480c1a0cc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.109491 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.109472    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.109576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.109518    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.109576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.109533    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.111636 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.111550    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5" pod "452532ae-6a10-49ca-85dd-c9480c1a0cc8" (UID: "452532ae-6a10-49ca-85dd-c9480c1a0cc8"). InnerVolumeSpecName "kube-api-access-sq6q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:28.111787 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.111762    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm" pod "452532ae-6a10-49ca-85dd-c9480c1a0cc8" (UID: "452532ae-6a10-49ca-85dd-c9480c1a0cc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.111888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.111865    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs" pod "452532ae-6a10-49ca-85dd-c9480c1a0cc8" (UID: "452532ae-6a10-49ca-85dd-c9480c1a0cc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:28.124377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.124316    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:23:28.128252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.128228    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:23:28.209944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.209912    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm\") pod \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") "
Oct 30 20:23:28.210138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.209953    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache\") pod \"42f2aa64-eccc-470b-8500-a0b0a2440c41\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") "
Oct 30 20:23:28.210138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.209987    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll\" (UniqueName: \"kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll\") pod \"42f2aa64-eccc-470b-8500-a0b0a2440c41\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") "
Oct 30 20:23:28.210138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210027    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir\") pod \"42f2aa64-eccc-470b-8500-a0b0a2440c41\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") "
Oct 30 20:23:28.210138 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210132    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs\" (UniqueName: \"kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs\") pod \"42f2aa64-eccc-470b-8500-a0b0a2440c41\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") "
Oct 30 20:23:28.210392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210225    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir\") pod \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") "
Oct 30 20:23:28.210392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210261    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home\") pod \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") "
Oct 30 20:23:28.210392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210312    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home\") pod \"42f2aa64-eccc-470b-8500-a0b0a2440c41\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") "
Oct 30 20:23:28.210392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210327    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir" pod "42f2aa64-eccc-470b-8500-a0b0a2440c41" (UID: "42f2aa64-eccc-470b-8500-a0b0a2440c41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.210392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210343    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache" pod "42f2aa64-eccc-470b-8500-a0b0a2440c41" (UID: "42f2aa64-eccc-470b-8500-a0b0a2440c41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.210392 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210354    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs\") pod \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") "
Oct 30 20:23:28.210679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210414    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache\") pod \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") "
Oct 30 20:23:28.210679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210448    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr\" (UniqueName: \"kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr\") pod \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\" (UID: \"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65\") "
Oct 30 20:23:28.210679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210487    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm\") pod \"42f2aa64-eccc-470b-8500-a0b0a2440c41\" (UID: \"42f2aa64-eccc-470b-8500-a0b0a2440c41\") "
Oct 30 20:23:28.210679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210574    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir" pod "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" (UID: "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.210679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210586    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home" pod "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" (UID: "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.210933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210707    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home" pod "42f2aa64-eccc-470b-8500-a0b0a2440c41" (UID: "42f2aa64-eccc-470b-8500-a0b0a2440c41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.211019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.210990    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452532ae-6a10-49ca-85dd-c9480c1a0cc8-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211073 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211019    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211073 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211035    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211073 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211050    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211073 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211063    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211079    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452532ae-6a10-49ca-85dd-c9480c1a0cc8-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211093    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211108    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq6q5\" (UniqueName: \"kubernetes.io/projected/452532ae-6a10-49ca-85dd-c9480c1a0cc8-kube-api-access-sq6q5\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.211240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.211030    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache" pod "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" (UID: "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.213008 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.212874    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm" pod "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" (UID: "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.213008 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.212877    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs" pod "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" (UID: "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:28.213008 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.212967    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs" pod "42f2aa64-eccc-470b-8500-a0b0a2440c41" (UID: "42f2aa64-eccc-470b-8500-a0b0a2440c41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:28.213273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.213059    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll" pod "42f2aa64-eccc-470b-8500-a0b0a2440c41" (UID: "42f2aa64-eccc-470b-8500-a0b0a2440c41"). InnerVolumeSpecName "kube-api-access-hvzll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:28.213273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.213074    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr" pod "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" (UID: "8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65"). InnerVolumeSpecName "kube-api-access-wnkdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:28.214031 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.214011    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm" pod "42f2aa64-eccc-470b-8500-a0b0a2440c41" (UID: "42f2aa64-eccc-470b-8500-a0b0a2440c41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:28.312414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312379    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.312414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312410    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.312414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312421    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnkdr\" (UniqueName: \"kubernetes.io/projected/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-kube-api-access-wnkdr\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.312878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312432    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42f2aa64-eccc-470b-8500-a0b0a2440c41-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.312878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312441    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.312878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312448    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvzll\" (UniqueName: \"kubernetes.io/projected/42f2aa64-eccc-470b-8500-a0b0a2440c41-kube-api-access-hvzll\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.312878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.312459    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42f2aa64-eccc-470b-8500-a0b0a2440c41-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:28.480902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.480862    6389 generic.go:356] "Generic (PLEG): container finished" podID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerID="00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa" exitCode=0
Oct 30 20:23:28.480902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.480904    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"
Oct 30 20:23:28.481181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.480954    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" event={"ID":"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65","Type":"ContainerDied","Data":"00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa"}
Oct 30 20:23:28.481181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.480996    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j" event={"ID":"8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65","Type":"ContainerDied","Data":"eca93c18264f87f469fc44ed1360ba66d1f83ba1b210b0c72961f75936a4009f"}
Oct 30 20:23:28.481181 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.481018    6389 scope.go:122] "RemoveContainer" containerID="00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa"
Oct 30 20:23:28.482368 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.482348    6389 generic.go:356] "Generic (PLEG): container finished" podID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerID="e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002" exitCode=0
Oct 30 20:23:28.482470 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.482452    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"
Oct 30 20:23:28.482470 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.482460    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" event={"ID":"452532ae-6a10-49ca-85dd-c9480c1a0cc8","Type":"ContainerDied","Data":"e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002"}
Oct 30 20:23:28.482586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.482490    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm" event={"ID":"452532ae-6a10-49ca-85dd-c9480c1a0cc8","Type":"ContainerDied","Data":"e32b129b8634059cdf69455e95eb4f2dd3ef9cae0db7777afa3f9da6618b8631"}
Oct 30 20:23:28.483944 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.483923    6389 generic.go:356] "Generic (PLEG): container finished" podID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerID="ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab" exitCode=0
Oct 30 20:23:28.484045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.483951    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" event={"ID":"42f2aa64-eccc-470b-8500-a0b0a2440c41","Type":"ContainerDied","Data":"ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab"}
Oct 30 20:23:28.484045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.483971    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b" event={"ID":"42f2aa64-eccc-470b-8500-a0b0a2440c41","Type":"ContainerDied","Data":"37f47ae67010b6b91ac47f82f3ed994fef9b9664a4c85fa95bad74ec0d45907b"}
Oct 30 20:23:28.484045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.484025    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"
Oct 30 20:23:28.491669 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.491656    6389 scope.go:122] "RemoveContainer" containerID="00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa"
Oct 30 20:23:28.492106 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:28.491952    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa\": container with ID starting with 00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa not found: ID does not exist" containerID="00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa"
Oct 30 20:23:28.492106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.491997    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa"} err="failed to get container status \"00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa\": rpc error: code = NotFound desc = could not find container \"00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa\": container with ID starting with 00bbf2edbd6a253b8dec46382205f517665c31a9814d7cc2839530d6e9c9effa not found: ID does not exist"
Oct 30 20:23:28.492106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.492020    6389 scope.go:122] "RemoveContainer" containerID="e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002"
Oct 30 20:23:28.518833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.518576    6389 scope.go:122] "RemoveContainer" containerID="e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002"
Oct 30 20:23:28.519379 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:28.519023    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002\": container with ID starting with e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002 not found: ID does not exist" containerID="e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002"
Oct 30 20:23:28.519379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.519076    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002"} err="failed to get container status \"e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002\": rpc error: code = NotFound desc = could not find container \"e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002\": container with ID starting with e721af3ee5b6cf4b3273ed4a96092e5a68150e2ed911ac71822f1466bbd48002 not found: ID does not exist"
Oct 30 20:23:28.519379 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.519097    6389 scope.go:122] "RemoveContainer" containerID="ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab"
Oct 30 20:23:28.529407 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.529387    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"]
Oct 30 20:23:28.531161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.531137    6389 scope.go:122] "RemoveContainer" containerID="ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab"
Oct 30 20:23:28.531483 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:28.531464    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab\": container with ID starting with ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab not found: ID does not exist" containerID="ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab"
Oct 30 20:23:28.531559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.531495    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab"} err="failed to get container status \"ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab\": rpc error: code = NotFound desc = could not find container \"ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab\": container with ID starting with ae3ff7dcf9ec9d6e5f61dce61dc8745afd0b722e1d0150e8efb554518d9b07ab not found: ID does not exist"
Oct 30 20:23:28.535057 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.535034    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-2-w1-5197ec-kserve-77f8785557-x96nm"]
Oct 30 20:23:28.546996 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.546975    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"]
Oct 30 20:23:28.555328 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.555305    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/distinct-w1-5197ec-kserve-58647b6cdb-q2f7b"]
Oct 30 20:23:28.565703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.565683    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"]
Oct 30 20:23:28.570514 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.570496    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5197ec/simulator-w1-5197ec-kserve-597d8f7854-m5r5j"]
Oct 30 20:23:28.859563 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:28.859532    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:29.765601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:29.765558    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:29.913313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:29.913280    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" path="/var/lib/kubelet/pods/42f2aa64-eccc-470b-8500-a0b0a2440c41/volumes"
Oct 30 20:23:29.913711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:29.913697    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" path="/var/lib/kubelet/pods/452532ae-6a10-49ca-85dd-c9480c1a0cc8/volumes"
Oct 30 20:23:29.914056 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:29.914045    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" path="/var/lib/kubelet/pods/8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65/volumes"
Oct 30 20:23:30.536789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:30.536756    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:31.492800 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:31.492758    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:32.130607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:32.130493    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:32.369141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:32.368209    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:32.543972 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:32.543936    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:32.675427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:32.675379    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:33.154137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:33.153829    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:33.754160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:33.754127    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:34.556915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:34.556883    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:35.096925 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:35.096897    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:35.227707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:35.227673    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:35.594333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:35.594304    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:35.707400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:35.707356    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:36.282751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:36.282705    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:23:36.560467 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:36.560378    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:37.552977 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:37.552919    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:38.318003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:38.317953    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:39.098685 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.098645    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:39.806755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.806723    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"]
Oct 30 20:23:39.807411 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807393    6389 cpu_manager.go:414] "containerMap: removing container" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main"
Oct 30 20:23:39.807461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807414    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main"
Oct 30 20:23:39.807461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807440    6389 cpu_manager.go:414] "containerMap: removing container" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main"
Oct 30 20:23:39.807461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807448    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main"
Oct 30 20:23:39.807461 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807460    6389 cpu_manager.go:414] "containerMap: removing container" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main"
Oct 30 20:23:39.807586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807468    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main"
Oct 30 20:23:39.807586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807491    6389 cpu_manager.go:414] "containerMap: removing container" podUID="7463ae95-2cbb-4615-abd0-eba3df86c4b1" containerName="revoke-keys"
Oct 30 20:23:39.807586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807498    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7463ae95-2cbb-4615-abd0-eba3df86c4b1" containerName="revoke-keys"
Oct 30 20:23:39.807720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807596    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="452532ae-6a10-49ca-85dd-c9480c1a0cc8" containerName="main"
Oct 30 20:23:39.807720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807608    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="42f2aa64-eccc-470b-8500-a0b0a2440c41" containerName="main"
Oct 30 20:23:39.807720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807623    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="7463ae95-2cbb-4615-abd0-eba3df86c4b1" containerName="revoke-keys"
Oct 30 20:23:39.807720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.807634    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dbbcef9-bccf-4cfe-abe8-0a0f9cea4a65" containerName="main"
Oct 30 20:23:39.812888 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.812867    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.818799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.818780    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-124fd1-openshift-default-dockercfg-n5plt\""
Oct 30 20:23:39.828456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.828427    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"]
Oct 30 20:23:39.882217 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882179    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882231    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882330    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtcwd\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882365    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882438    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882479    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882556 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882507    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882556 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882549    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.882620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.882590    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.983439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983402    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.983649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983465    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.983649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983509    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtcwd\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.983649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983543    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.983649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983575    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.983649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983622    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983670    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983700    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983769    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983855    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.983896    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.984133    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.984398    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.984535 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.984519    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.986103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.986077    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:39.986215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:39.986201    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:40.008168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.008113    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:40.010174 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.010153    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtcwd\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd\") pod \"e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") " pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:40.129941 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.129856    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-124fd1-openshift-default-dockercfg-n5plt\""
Oct 30 20:23:40.137830 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.137806    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:40.438189 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.437024    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"]
Oct 30 20:23:40.443775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.443732    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:23:40.443881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.443816    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:23:40.443881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.443857    6389 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:23:40.466961 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.466877    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:40.590443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.590401    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" event={"ID":"89024f8b-846e-4309-94c8-1f0e1b05af2a","Type":"ContainerStarted","Data":"60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0"}
Oct 30 20:23:40.590625 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.590463    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" event={"ID":"89024f8b-846e-4309-94c8-1f0e1b05af2a","Type":"ContainerStarted","Data":"4364a70c51dffa6582942b58c7d1431f9e218e8638bf98ae9129ffb2961200bf"}
Oct 30 20:23:40.617549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:40.617495    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podStartSLOduration=1.617482359 podStartE2EDuration="1.617482359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:23:40.614185262 +0000 UTC m=+1140.813758191" watchObservedRunningTime="2026-10-30 20:23:40.617482359 +0000 UTC m=+1140.817055286"
Oct 30 20:23:41.141751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.140263    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:41.146144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.144660    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:41.146144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.144723    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:41.309349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.309312    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:41.419222 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.419113    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:41.534230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.534196    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:41.677496 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.677392    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"]
Oct 30 20:23:41.677669 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.677644    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy" containerID="cri-o://ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e" gracePeriod=30
Oct 30 20:23:41.682533 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.682503    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"]
Oct 30 20:23:41.682783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.682752    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy" containerID="cri-o://4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31" gracePeriod=30
Oct 30 20:23:41.701253 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.701228    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"]
Oct 30 20:23:41.701477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.701428    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" containerID="cri-o://b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3" gracePeriod=30
Oct 30 20:23:41.704405 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.704382    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"]
Oct 30 20:23:41.704587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.704563    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy" containerID="cri-o://9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb" gracePeriod=30
Oct 30 20:23:41.774144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:41.769500    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:42.139142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:42.138486    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:42.139142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:42.138553    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:42.446705 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:42.446594    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:43.138889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.138433    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:43.138889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.138500    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:43.358158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.358105    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:43.409627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.409602    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:23:43.519718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.519682    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume\") pod \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") "
Oct 30 20:23:43.520105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.519736    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z\" (UniqueName: \"kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z\") pod \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\" (UID: \"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb\") "
Oct 30 20:23:43.520105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.520019    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume" pod "d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" (UID: "d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:43.520228 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.520144    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:43.546577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.546543    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z" pod "d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" (UID: "d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb"). InnerVolumeSpecName "kube-api-access-mzm6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:43.611298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.611265    6389 generic.go:356] "Generic (PLEG): container finished" podID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerID="a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc" exitCode=0
Oct 30 20:23:43.611472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.611321    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"
Oct 30 20:23:43.611472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.611332    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" event={"ID":"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb","Type":"ContainerDied","Data":"a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc"}
Oct 30 20:23:43.611472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.611406    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz" event={"ID":"d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb","Type":"ContainerDied","Data":"352ec395914fc3cad0a526347583514c68cd297a0506386ce45112f1ba549edf"}
Oct 30 20:23:43.611472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.611428    6389 scope.go:122] "RemoveContainer" containerID="a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc"
Oct 30 20:23:43.621036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.621017    6389 scope.go:122] "RemoveContainer" containerID="a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc"
Oct 30 20:23:43.621184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.621082    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzm6z\" (UniqueName: \"kubernetes.io/projected/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb-kube-api-access-mzm6z\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:43.646346 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:43.646263    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc\": container with ID starting with a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc not found: ID does not exist" containerID="a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc"
Oct 30 20:23:43.646346 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.646312    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc"} err="failed to get container status \"a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc\": rpc error: code = NotFound desc = could not find container \"a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc\": container with ID starting with a4ae66a9393aa2c31bd8a05a99e8cb7ffa9a9d3727ee7aa85c18e194b31807cc not found: ID does not exist"
Oct 30 20:23:43.670377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.670350    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"]
Oct 30 20:23:43.675779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.675755    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-5197ec-6fb9b4c8fb-r45tz"]
Oct 30 20:23:43.913474 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:43.913402    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" path="/var/lib/kubelet/pods/d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb/volumes"
Oct 30 20:23:44.145141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:44.144539    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:44.145141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:44.144602    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:44.247144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:44.242075    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:45.143557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:45.143520    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:45.144135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:45.144083    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:45.184771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:45.184740    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:46.013562 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.013528    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:46.147520 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.147480    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:46.148015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.147535    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:46.547646 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.547607    6389 patch_prober.go:29] interesting pod/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:23:46.547809 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.547677    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:23:46.863112 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.863051    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:46.971101 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.971074    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:23:46.994462 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:46.994432    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:23:47.032161 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.032113    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:23:47.076061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.075995    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076034    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.076301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076071    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076139    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076185    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076218    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.076301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076248    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.076301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076279    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076324    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076370    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076396    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076425    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076453    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076522    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076531    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.076620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076591    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076623    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076661    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076693    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token\") pod \"cb503944-3ca8-4861-a51d-b1e2448ef651\" (UID: \"cb503944-3ca8-4861-a51d-b1e2448ef651\") "
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076742    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy\") pod \"c1125734-af65-4eb1-a99b-887888ea4e6c\" (UID: \"c1125734-af65-4eb1-a99b-887888ea4e6c\") "
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076745    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076948    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.077038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.076964    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.077424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077231    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.077424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077254    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.077424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077269    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.077424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077285    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.077424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077299    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.077729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077621    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:47.078076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.077959    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:47.078898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.078871    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.079142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.079045    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.079233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.079145    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.079578 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.079553    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "kube-api-access-tvvbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.081325 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.081298    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.081325 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.081311    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.081469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.081407    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.081469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.081410    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:47.081469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.081433    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "kube-api-access-4cnd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.081571 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.081520    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token" pod "c1125734-af65-4eb1-a99b-887888ea4e6c" (UID: "c1125734-af65-4eb1-a99b-887888ea4e6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.082623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.082601    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo" pod "cb503944-3ca8-4861-a51d-b1e2448ef651" (UID: "cb503944-3ca8-4861-a51d-b1e2448ef651"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:47.139136 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.139095    6389 patch_prober.go:29] interesting pod/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 20:23:47.139275 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.139187    6389 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 20:23:47.178085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178051    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178092    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178134    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178165    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178234    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178260    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178327    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178393    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178398    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178421    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert\") pod \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\" (UID: \"e430a09e-18f3-471a-a53b-a0fcf7b8e626\") "
Oct 30 20:23:47.178572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178481    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178650    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178723    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178898    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.178980    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179001    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cnd6\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-kube-api-access-4cnd6\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179015    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179028    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179041    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1125734-af65-4eb1-a99b-887888ea4e6c-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179057    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179072    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179085    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb503944-3ca8-4861-a51d-b1e2448ef651-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179099    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179113    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179150    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179165    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb503944-3ca8-4861-a51d-b1e2448ef651-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179180    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179195    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvvbm\" (UniqueName: \"kubernetes.io/projected/c1125734-af65-4eb1-a99b-887888ea4e6c-kube-api-access-tvvbm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179209    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1125734-af65-4eb1-a99b-887888ea4e6c-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179224    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.179754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.179241    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.180546 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.180520    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.180652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.180636    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "kube-api-access-6vpkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.180722 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.180708    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:47.181149 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.181101    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token" pod "e430a09e-18f3-471a-a53b-a0fcf7b8e626" (UID: "e430a09e-18f3-471a-a53b-a0fcf7b8e626"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.280795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.280758    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.280795 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.280797    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpkn\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-kube-api-access-6vpkn\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.281039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.280811    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.281039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.280825    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.281039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.280837    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e430a09e-18f3-471a-a53b-a0fcf7b8e626-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.349889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.349860    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:23:47.482439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482409    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482451    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482484    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482524    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482583    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482619    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482664    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482710    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482705    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.482965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482725    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.482965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482758    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token\") pod \"6250214c-7743-4a40-bac4-152a3349becd\" (UID: \"6250214c-7743-4a40-bac4-152a3349becd\") "
Oct 30 20:23:47.482965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.482908    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.483216 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.483083    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:47.483273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.483212    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.483273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.483236    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.483273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.483252    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.484144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.484093    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.485430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.485397    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.486035 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.486005    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "kube-api-access-lrqlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:47.486113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.486043    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:47.487594 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.487575    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy" pod "6250214c-7743-4a40-bac4-152a3349becd" (UID: "6250214c-7743-4a40-bac4-152a3349becd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:47.583973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.583936    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6250214c-7743-4a40-bac4-152a3349becd-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.583973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.583967    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.584233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.583981    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6250214c-7743-4a40-bac4-152a3349becd-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.584233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.583995    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrqlq\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-kube-api-access-lrqlq\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.584233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.584009    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6250214c-7743-4a40-bac4-152a3349becd-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.584233 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.584021    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6250214c-7743-4a40-bac4-152a3349becd-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:47.645261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.645167    6389 generic.go:356] "Generic (PLEG): container finished" podID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerID="b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3" exitCode=0
Oct 30 20:23:47.645261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.645226    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"
Oct 30 20:23:47.645484 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.645259    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" event={"ID":"c1125734-af65-4eb1-a99b-887888ea4e6c","Type":"ContainerDied","Data":"b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3"}
Oct 30 20:23:47.645484 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.645360    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj" event={"ID":"c1125734-af65-4eb1-a99b-887888ea4e6c","Type":"ContainerDied","Data":"a0c0af6375f8bc9234723eee5d237eb61ea6c3850423725b330f4e9c17038dd8"}
Oct 30 20:23:47.645484 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.645393    6389 scope.go:122] "RemoveContainer" containerID="b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3"
Oct 30 20:23:47.646988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.646967    6389 generic.go:356] "Generic (PLEG): container finished" podID="6250214c-7743-4a40-bac4-152a3349becd" containerID="4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31" exitCode=0
Oct 30 20:23:47.647297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.647001    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" event={"ID":"6250214c-7743-4a40-bac4-152a3349becd","Type":"ContainerDied","Data":"4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31"}
Oct 30 20:23:47.647297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.647259    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c" event={"ID":"6250214c-7743-4a40-bac4-152a3349becd","Type":"ContainerDied","Data":"d1532d8aeb19dacd227d43ce3ad91c5203fcf6d45dc2c97bcdfcad0df9737f50"}
Oct 30 20:23:47.647297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.647024    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"
Oct 30 20:23:47.648916 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.648890    6389 generic.go:356] "Generic (PLEG): container finished" podID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerID="ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e" exitCode=0
Oct 30 20:23:47.649071 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.649052    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" event={"ID":"cb503944-3ca8-4861-a51d-b1e2448ef651","Type":"ContainerDied","Data":"ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e"}
Oct 30 20:23:47.649173 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.649098    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw" event={"ID":"cb503944-3ca8-4861-a51d-b1e2448ef651","Type":"ContainerDied","Data":"83f3de7200a62b5682e9f1476d9e009a7fa8c1963410452c075321ad2209c931"}
Oct 30 20:23:47.649238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.649215    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"
Oct 30 20:23:47.662940 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.662911    6389 generic.go:356] "Generic (PLEG): container finished" podID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerID="9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb" exitCode=0
Oct 30 20:23:47.663030 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.662997    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" event={"ID":"e430a09e-18f3-471a-a53b-a0fcf7b8e626","Type":"ContainerDied","Data":"9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb"}
Oct 30 20:23:47.663076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.663038    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5" event={"ID":"e430a09e-18f3-471a-a53b-a0fcf7b8e626","Type":"ContainerDied","Data":"8ea7d0acef3ca9986c77539c6e03150401f5f2a3973272195e2b88ddcd830dfe"}
Oct 30 20:23:47.663147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.663106    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"
Oct 30 20:23:47.664004 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.663973    6389 scope.go:122] "RemoveContainer" containerID="b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3"
Oct 30 20:23:47.666941 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:47.666917    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3\": container with ID starting with b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3 not found: ID does not exist" containerID="b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3"
Oct 30 20:23:47.667046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.666950    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3"} err="failed to get container status \"b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3\": rpc error: code = NotFound desc = could not find container \"b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3\": container with ID starting with b86e28790e2844cb6bd6e95df2fda3cd4c150f331c879e145751c6007509c9a3 not found: ID does not exist"
Oct 30 20:23:47.667046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.666974    6389 scope.go:122] "RemoveContainer" containerID="4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31"
Oct 30 20:23:47.677995 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.677980    6389 scope.go:122] "RemoveContainer" containerID="4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31"
Oct 30 20:23:47.678335 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:47.678316    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31\": container with ID starting with 4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31 not found: ID does not exist" containerID="4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31"
Oct 30 20:23:47.678427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.678344    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31"} err="failed to get container status \"4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31\": rpc error: code = NotFound desc = could not find container \"4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31\": container with ID starting with 4200c296bea50f32ec69b6bb3786c1ec7dc22cf396aca99979f889867e79aa31 not found: ID does not exist"
Oct 30 20:23:47.678427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.678366    6389 scope.go:122] "RemoveContainer" containerID="ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e"
Oct 30 20:23:47.684445 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.684413    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"]
Oct 30 20:23:47.688909 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.688886    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-7k55c"]
Oct 30 20:23:47.696393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.696370    6389 scope.go:122] "RemoveContainer" containerID="ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e"
Oct 30 20:23:47.696831 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:47.696711    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e\": container with ID starting with ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e not found: ID does not exist" containerID="ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e"
Oct 30 20:23:47.696831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.696745    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e"} err="failed to get container status \"ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e\": rpc error: code = NotFound desc = could not find container \"ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e\": container with ID starting with ed3b0b886c18f5c5401d9ac56ccc6c17b23cd8edfe2f413002d0e6c7136b925e not found: ID does not exist"
Oct 30 20:23:47.696831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.696769    6389 scope.go:122] "RemoveContainer" containerID="9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb"
Oct 30 20:23:47.710238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.710210    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"]
Oct 30 20:23:47.716522 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.716499    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-kg2vj"]
Oct 30 20:23:47.723204 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.723180    6389 scope.go:122] "RemoveContainer" containerID="9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb"
Oct 30 20:23:47.723513 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:47.723493    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb\": container with ID starting with 9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb not found: ID does not exist" containerID="9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb"
Oct 30 20:23:47.723610 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.723519    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb"} err="failed to get container status \"9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb\": rpc error: code = NotFound desc = could not find container \"9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb\": container with ID starting with 9d151f7c6c2740e9d12ab177e113e0d9ba81fc5c2d575de3db84fa3f26fe06cb not found: ID does not exist"
Oct 30 20:23:47.731947 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.731890    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"]
Oct 30 20:23:47.736656 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.736632    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-wj4j5"]
Oct 30 20:23:47.749724 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.749702    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"]
Oct 30 20:23:47.755903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.755881    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5197ec-openshift-default-7b9b67ff59-4v9lw"]
Oct 30 20:23:47.913003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.912916    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6250214c-7743-4a40-bac4-152a3349becd" path="/var/lib/kubelet/pods/6250214c-7743-4a40-bac4-152a3349becd/volumes"
Oct 30 20:23:47.913469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.913448    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" path="/var/lib/kubelet/pods/c1125734-af65-4eb1-a99b-887888ea4e6c/volumes"
Oct 30 20:23:47.913908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.913890    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" path="/var/lib/kubelet/pods/cb503944-3ca8-4861-a51d-b1e2448ef651/volumes"
Oct 30 20:23:47.914682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.914662    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" path="/var/lib/kubelet/pods/e430a09e-18f3-471a-a53b-a0fcf7b8e626/volumes"
Oct 30 20:23:47.958029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:47.958001    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:48.142287 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:48.142250    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:48.142632 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:48.142614    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:48.143406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:48.143385    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:23:48.748038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:48.747998    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:49.802301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:49.802267    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:50.524551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:50.524521    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:50.967039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:50.967008    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:51.070077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:51.070047    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:51.570424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:51.570386    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:51.943662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:51.943572    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:52.470167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:52.470101    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:53.176454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:53.176420    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:54.290675 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:54.290641    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:55.187280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:55.187245    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:55.792688 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:23:55.792641    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:23:55.939740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:55.939702    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"]
Oct 30 20:23:55.946055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:55.946027    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"]
Oct 30 20:23:56.038800 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:56.038757    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:56.142145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:56.142050    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:56.873715 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:56.873686    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:57.586859 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:57.586816    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:58.449298 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:58.449264    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:58.968275 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:58.968245    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:59.187296 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:59.187262    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:23:59.890166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:23:59.890128    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:00.776716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:00.776681    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:01.264501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:01.264469    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"]
Oct 30 20:24:01.267426 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:01.267400    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"]
Oct 30 20:24:01.351772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:01.351738    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:01.452721 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:01.452686    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:01.561183 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:01.561073    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:02.183599 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.183559    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"]
Oct 30 20:24:02.184206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184186    6389 cpu_manager.go:414] "containerMap: removing container" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy"
Oct 30 20:24:02.184265 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184210    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy"
Oct 30 20:24:02.184265 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184235    6389 cpu_manager.go:414] "containerMap: removing container" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy"
Oct 30 20:24:02.184265 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184244    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy"
Oct 30 20:24:02.184362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184275    6389 cpu_manager.go:414] "containerMap: removing container" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerName="payload-processing"
Oct 30 20:24:02.184362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184283    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerName="payload-processing"
Oct 30 20:24:02.184362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184296    6389 cpu_manager.go:414] "containerMap: removing container" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy"
Oct 30 20:24:02.184362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184304    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy"
Oct 30 20:24:02.184362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184330    6389 cpu_manager.go:414] "containerMap: removing container" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy"
Oct 30 20:24:02.184362 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184338    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy"
Oct 30 20:24:02.184566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184430    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb503944-3ca8-4861-a51d-b1e2448ef651" containerName="istio-proxy"
Oct 30 20:24:02.184566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184445    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1125734-af65-4eb1-a99b-887888ea4e6c" containerName="istio-proxy"
Oct 30 20:24:02.184566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184458    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="6250214c-7743-4a40-bac4-152a3349becd" containerName="istio-proxy"
Oct 30 20:24:02.184566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184471    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9d15912-69fe-4e8f-8e0c-2634a8f6e4cb" containerName="payload-processing"
Oct 30 20:24:02.184566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.184482    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="e430a09e-18f3-471a-a53b-a0fcf7b8e626" containerName="istio-proxy"
Oct 30 20:24:02.190711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.190687    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.199592 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.199559    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"]
Oct 30 20:24:02.288218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.288184    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:24:02.288218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.288222    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:24:02.333332 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.333295    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.333490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.333433    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.333539 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.333481    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6ksd\" (UniqueName: \"kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.434468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.434365    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.434642 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.434477    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.434642 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.434515    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6ksd\" (UniqueName: \"kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.435419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.435384    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.436925 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.436892    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.448320 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.448287    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6ksd\" (UniqueName: \"kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd\") pod \"maas-api-revoke-sub-73dfb91d-5jrtq\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.514257 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.514223    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:02.800880 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:02.800850    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"]
Oct 30 20:24:02.802761 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:24:02.802738    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719f5dee_683a_4a32_bdf6_42280a157252.slice/crio-adde47b3d45211997bace57b73625c729d63774e7a7bdd1b94cb3be417876ebf WatchSource:0}: Error finding container adde47b3d45211997bace57b73625c729d63774e7a7bdd1b94cb3be417876ebf: Status 404 returned error can't find the container with id adde47b3d45211997bace57b73625c729d63774e7a7bdd1b94cb3be417876ebf
Oct 30 20:24:03.764218 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:03.764192    6389 generic.go:356] "Generic (PLEG): container finished" podID="719f5dee-683a-4a32-bdf6-42280a157252" containerID="ccbc14ad82c37e0bf27046313909cf61494ae26cd4a885da85e5ebc31ca067d4" exitCode=0
Oct 30 20:24:03.764594 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:03.764228    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq" event={"ID":"719f5dee-683a-4a32-bdf6-42280a157252","Type":"ContainerDied","Data":"ccbc14ad82c37e0bf27046313909cf61494ae26cd4a885da85e5ebc31ca067d4"}
Oct 30 20:24:03.764594 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:03.764278    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq" event={"ID":"719f5dee-683a-4a32-bdf6-42280a157252","Type":"ContainerStarted","Data":"adde47b3d45211997bace57b73625c729d63774e7a7bdd1b94cb3be417876ebf"}
Oct 30 20:24:04.898212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:04.898150    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:05.060225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.060138    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca\") pod \"719f5dee-683a-4a32-bdf6-42280a157252\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") "
Oct 30 20:24:05.060225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.060188    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca\") pod \"719f5dee-683a-4a32-bdf6-42280a157252\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") "
Oct 30 20:24:05.060465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.060233    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd\" (UniqueName: \"kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd\") pod \"719f5dee-683a-4a32-bdf6-42280a157252\" (UID: \"719f5dee-683a-4a32-bdf6-42280a157252\") "
Oct 30 20:24:05.060565 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.060539    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca" pod "719f5dee-683a-4a32-bdf6-42280a157252" (UID: "719f5dee-683a-4a32-bdf6-42280a157252"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:05.062244 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.062221    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca" pod "719f5dee-683a-4a32-bdf6-42280a157252" (UID: "719f5dee-683a-4a32-bdf6-42280a157252"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:05.062385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.062371    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd" pod "719f5dee-683a-4a32-bdf6-42280a157252" (UID: "719f5dee-683a-4a32-bdf6-42280a157252"). InnerVolumeSpecName "kube-api-access-s6ksd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:05.161644 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.161613    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:05.161644 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.161641    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/719f5dee-683a-4a32-bdf6-42280a157252-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:05.161799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.161652    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6ksd\" (UniqueName: \"kubernetes.io/projected/719f5dee-683a-4a32-bdf6-42280a157252-kube-api-access-s6ksd\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:05.777876 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.777845    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"
Oct 30 20:24:05.778055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.777844    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq" event={"ID":"719f5dee-683a-4a32-bdf6-42280a157252","Type":"ContainerDied","Data":"adde47b3d45211997bace57b73625c729d63774e7a7bdd1b94cb3be417876ebf"}
Oct 30 20:24:05.778055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:05.777969    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adde47b3d45211997bace57b73625c729d63774e7a7bdd1b94cb3be417876ebf"
Oct 30 20:24:06.265210 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:06.265168    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" containerName="payload-processing" containerID="cri-o://c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466" gracePeriod=25
Oct 30 20:24:06.267636 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:06.267605    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerName="payload-pre-processing" containerID="cri-o://3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b" gracePeriod=25
Oct 30 20:24:07.285216 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:07.285175    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:24:07.285596 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:07.285213    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:24:10.961387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:10.961345    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main" containerID="cri-o://bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa" gracePeriod=45
Oct 30 20:24:10.966557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:10.966524    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main" containerID="cri-o://24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4" gracePeriod=45
Oct 30 20:24:11.222605 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.222579    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:24:11.225393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.225369    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:24:11.314437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314406    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm\") pod \"831bc213-3f56-42f5-88a2-12d399c1a81e\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") "
Oct 30 20:24:11.314626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314448    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs\" (UniqueName: \"kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs\") pod \"831bc213-3f56-42f5-88a2-12d399c1a81e\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") "
Oct 30 20:24:11.314626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314498    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs\" (UniqueName: \"kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs\") pod \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") "
Oct 30 20:24:11.314626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314517    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8\" (UniqueName: \"kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8\") pod \"831bc213-3f56-42f5-88a2-12d399c1a81e\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") "
Oct 30 20:24:11.314626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314538    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir\") pod \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") "
Oct 30 20:24:11.314626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314555    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache\") pod \"831bc213-3f56-42f5-88a2-12d399c1a81e\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") "
Oct 30 20:24:11.314626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314594    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg\" (UniqueName: \"kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg\") pod \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") "
Oct 30 20:24:11.314959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314650    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir\") pod \"831bc213-3f56-42f5-88a2-12d399c1a81e\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") "
Oct 30 20:24:11.314959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314717    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home\") pod \"831bc213-3f56-42f5-88a2-12d399c1a81e\" (UID: \"831bc213-3f56-42f5-88a2-12d399c1a81e\") "
Oct 30 20:24:11.314959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314749    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache\") pod \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") "
Oct 30 20:24:11.314959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314824    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home\") pod \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") "
Oct 30 20:24:11.314959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314830    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir" pod "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" (UID: "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.314959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.314871    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm\") pod \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\" (UID: \"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b\") "
Oct 30 20:24:11.315285 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.315222    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.315469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.315440    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home" pod "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" (UID: "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.315576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.315507    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache" pod "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" (UID: "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.315910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.315841    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir" pod "831bc213-3f56-42f5-88a2-12d399c1a81e" (UID: "831bc213-3f56-42f5-88a2-12d399c1a81e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.315910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.315804    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home" pod "831bc213-3f56-42f5-88a2-12d399c1a81e" (UID: "831bc213-3f56-42f5-88a2-12d399c1a81e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.315910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.315851    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache" pod "831bc213-3f56-42f5-88a2-12d399c1a81e" (UID: "831bc213-3f56-42f5-88a2-12d399c1a81e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.317645 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.317620    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm" pod "831bc213-3f56-42f5-88a2-12d399c1a81e" (UID: "831bc213-3f56-42f5-88a2-12d399c1a81e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.317983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.317961    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs" pod "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" (UID: "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:11.319192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.319140    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm" pod "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" (UID: "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:11.319281 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.319221    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs" pod "831bc213-3f56-42f5-88a2-12d399c1a81e" (UID: "831bc213-3f56-42f5-88a2-12d399c1a81e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:11.319497 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.319470    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg" pod "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" (UID: "cf1c2b2c-022e-4125-8d0f-112af9b9ee5b"). InnerVolumeSpecName "kube-api-access-tgdkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:11.319771 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.319735    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8" pod "831bc213-3f56-42f5-88a2-12d399c1a81e" (UID: "831bc213-3f56-42f5-88a2-12d399c1a81e"). InnerVolumeSpecName "kube-api-access-4twl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:11.416587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416543    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416582    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416591    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/831bc213-3f56-42f5-88a2-12d399c1a81e-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416600    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416609    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4twl8\" (UniqueName: \"kubernetes.io/projected/831bc213-3f56-42f5-88a2-12d399c1a81e-kube-api-access-4twl8\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416620    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416629    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgdkg\" (UniqueName: \"kubernetes.io/projected/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-kube-api-access-tgdkg\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416638    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416646    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/831bc213-3f56-42f5-88a2-12d399c1a81e-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416653    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.416869 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.416661    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:11.813267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.813235    6389 generic.go:356] "Generic (PLEG): container finished" podID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerID="bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa" exitCode=0
Oct 30 20:24:11.813454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.813290    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"
Oct 30 20:24:11.813454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.813324    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" event={"ID":"831bc213-3f56-42f5-88a2-12d399c1a81e","Type":"ContainerDied","Data":"bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa"}
Oct 30 20:24:11.813454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.813363    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k" event={"ID":"831bc213-3f56-42f5-88a2-12d399c1a81e","Type":"ContainerDied","Data":"97a0f1bb1d5d836b7cbe292e534211bbad88ad5c593afc41080afedfe29fa2f3"}
Oct 30 20:24:11.813454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.813384    6389 scope.go:122] "RemoveContainer" containerID="bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa"
Oct 30 20:24:11.814900 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.814875    6389 generic.go:356] "Generic (PLEG): container finished" podID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerID="24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4" exitCode=0
Oct 30 20:24:11.815015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.814912    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" event={"ID":"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b","Type":"ContainerDied","Data":"24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4"}
Oct 30 20:24:11.815015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.814927    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"
Oct 30 20:24:11.815015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.814940    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp" event={"ID":"cf1c2b2c-022e-4125-8d0f-112af9b9ee5b","Type":"ContainerDied","Data":"55c54d8e69f971fd540ec9b2f701cb925a25b92fd8f42f2295f76d490ddd8045"}
Oct 30 20:24:11.825099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.825079    6389 scope.go:122] "RemoveContainer" containerID="bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa"
Oct 30 20:24:11.825491 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:11.825474    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa\": container with ID starting with bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa not found: ID does not exist" containerID="bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa"
Oct 30 20:24:11.825559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.825498    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa"} err="failed to get container status \"bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa\": rpc error: code = NotFound desc = could not find container \"bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa\": container with ID starting with bb9b16086d2d2c04f6bcd9e6534df4d61dbbb5d7bd75a33d4beae425c9a25eaa not found: ID does not exist"
Oct 30 20:24:11.825559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.825514    6389 scope.go:122] "RemoveContainer" containerID="24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4"
Oct 30 20:24:11.833665 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.833644    6389 scope.go:122] "RemoveContainer" containerID="24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4"
Oct 30 20:24:11.833919 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:11.833898    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4\": container with ID starting with 24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4 not found: ID does not exist" containerID="24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4"
Oct 30 20:24:11.833970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.833924    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4"} err="failed to get container status \"24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4\": rpc error: code = NotFound desc = could not find container \"24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4\": container with ID starting with 24b4dafafa4bd0544f0a08c3a1c26ed3e29ed23d7ca93560b8ed60c3015403b4 not found: ID does not exist"
Oct 30 20:24:11.838857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.838838    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"]
Oct 30 20:24:11.844266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.844247    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/premium-w3-124fd1-kserve-6fffb4ff4f-hvq2k"]
Oct 30 20:24:11.856697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.856676    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"]
Oct 30 20:24:11.861844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.861823    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-124fd1/simulator-w3-124fd1-kserve-b58d7c57b-gztfp"]
Oct 30 20:24:11.912782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.912754    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" path="/var/lib/kubelet/pods/831bc213-3f56-42f5-88a2-12d399c1a81e/volumes"
Oct 30 20:24:11.913234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:11.913219    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" path="/var/lib/kubelet/pods/cf1c2b2c-022e-4125-8d0f-112af9b9ee5b/volumes"
Oct 30 20:24:13.764311 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:13.764264    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:14.779159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:14.779105    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:17.282769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:17.282732    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:17.285723 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:17.285683    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:24:17.285867 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:17.285683    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:24:17.449975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:17.449940    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:17.553905 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:17.553818    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:18.054024 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:18.053988    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:24.496742 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:24.496703    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:24.610919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:24.610890    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:24.850713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:24.850634    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"]
Oct 30 20:24:24.850882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:24.850833    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy" containerID="cri-o://60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0" gracePeriod=30
Oct 30 20:24:24.890075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:24.890043    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:25.791632 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:25.791600    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:24:26.510187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.510162    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:24:26.537260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.537238    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:24:26.555906 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.555880    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume\") pod \"b40e1118-0922-4fbc-a1db-02244272ce81\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") "
Oct 30 20:24:26.556022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.555995    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9\" (UniqueName: \"kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9\") pod \"b40e1118-0922-4fbc-a1db-02244272ce81\" (UID: \"b40e1118-0922-4fbc-a1db-02244272ce81\") "
Oct 30 20:24:26.556246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.556222    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume" pod "b40e1118-0922-4fbc-a1db-02244272ce81" (UID: "b40e1118-0922-4fbc-a1db-02244272ce81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:26.558055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.558029    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9" pod "b40e1118-0922-4fbc-a1db-02244272ce81" (UID: "b40e1118-0922-4fbc-a1db-02244272ce81"). InnerVolumeSpecName "kube-api-access-6xhz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:26.656780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.656755    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr\" (UniqueName: \"kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr\") pod \"4eb940e8-8ce9-4371-a732-0dcca00a0047\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") "
Oct 30 20:24:26.656898 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.656830    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume\") pod \"4eb940e8-8ce9-4371-a732-0dcca00a0047\" (UID: \"4eb940e8-8ce9-4371-a732-0dcca00a0047\") "
Oct 30 20:24:26.657075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.657044    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xhz9\" (UniqueName: \"kubernetes.io/projected/b40e1118-0922-4fbc-a1db-02244272ce81-kube-api-access-6xhz9\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:26.657075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.657073    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b40e1118-0922-4fbc-a1db-02244272ce81-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:26.657224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.657208    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume" pod "4eb940e8-8ce9-4371-a732-0dcca00a0047" (UID: "4eb940e8-8ce9-4371-a732-0dcca00a0047"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:26.658636 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.658622    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr" pod "4eb940e8-8ce9-4371-a732-0dcca00a0047" (UID: "4eb940e8-8ce9-4371-a732-0dcca00a0047"). InnerVolumeSpecName "kube-api-access-d47tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:26.757807 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.757782    6389 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb940e8-8ce9-4371-a732-0dcca00a0047-plugins-config-volume\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:26.757807 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.757806    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d47tr\" (UniqueName: \"kubernetes.io/projected/4eb940e8-8ce9-4371-a732-0dcca00a0047-kube-api-access-d47tr\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:26.908152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.908062    6389 generic.go:356] "Generic (PLEG): container finished" podID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerID="3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b" exitCode=0
Oct 30 20:24:26.908152 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.908103    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"
Oct 30 20:24:26.908618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.908149    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" event={"ID":"4eb940e8-8ce9-4371-a732-0dcca00a0047","Type":"ContainerDied","Data":"3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b"}
Oct 30 20:24:26.908618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.908206    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh" event={"ID":"4eb940e8-8ce9-4371-a732-0dcca00a0047","Type":"ContainerDied","Data":"52755b1ac530dd99f86e45bc9d31776e0c4e17741f96e17816c5c62be3eb8c4a"}
Oct 30 20:24:26.908618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.908237    6389 scope.go:122] "RemoveContainer" containerID="3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b"
Oct 30 20:24:26.909509 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.909489    6389 generic.go:356] "Generic (PLEG): container finished" podID="b40e1118-0922-4fbc-a1db-02244272ce81" containerID="c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466" exitCode=0
Oct 30 20:24:26.909600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.909541    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"
Oct 30 20:24:26.909600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.909581    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" event={"ID":"b40e1118-0922-4fbc-a1db-02244272ce81","Type":"ContainerDied","Data":"c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466"}
Oct 30 20:24:26.909684 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.909627    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr" event={"ID":"b40e1118-0922-4fbc-a1db-02244272ce81","Type":"ContainerDied","Data":"2889741660bd078e03e034cc163960480d3675857db2ce51e39be0231a640b66"}
Oct 30 20:24:26.918711 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.918689    6389 scope.go:122] "RemoveContainer" containerID="3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b"
Oct 30 20:24:26.919238 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:26.919205    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b\": container with ID starting with 3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b not found: ID does not exist" containerID="3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b"
Oct 30 20:24:26.919324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.919253    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b"} err="failed to get container status \"3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b\": rpc error: code = NotFound desc = could not find container \"3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b\": container with ID starting with 3ef346104b767655003766bffcdf64ccdfc1a21e1aeca04cd9c7df428ba5ba3b not found: ID does not exist"
Oct 30 20:24:26.919324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.919274    6389 scope.go:122] "RemoveContainer" containerID="c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466"
Oct 30 20:24:26.927772 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.927751    6389 scope.go:122] "RemoveContainer" containerID="c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466"
Oct 30 20:24:26.928020 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:26.928002    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466\": container with ID starting with c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466 not found: ID does not exist" containerID="c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466"
Oct 30 20:24:26.928075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.928027    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466"} err="failed to get container status \"c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466\": rpc error: code = NotFound desc = could not find container \"c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466\": container with ID starting with c1d8d505e93f195428f490ea2db0bdfd7a28b869773d8b6b99d018b8c9523466 not found: ID does not exist"
Oct 30 20:24:26.939857 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.939836    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"]
Oct 30 20:24:26.942252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.942234    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-124fd1-7fd767567-7gfhh"]
Oct 30 20:24:26.952707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.952682    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"]
Oct 30 20:24:26.954832 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:26.954815    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-124fd1-6d99d6b964-h8gkr"]
Oct 30 20:24:27.920401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:27.920364    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" path="/var/lib/kubelet/pods/4eb940e8-8ce9-4371-a732-0dcca00a0047/volumes"
Oct 30 20:24:27.920782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:27.920757    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" path="/var/lib/kubelet/pods/b40e1118-0922-4fbc-a1db-02244272ce81/volumes"
Oct 30 20:24:30.398788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.398704    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:24:30.508489 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508456    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508518    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508571    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508595    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508630    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508650    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508676    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508717    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.508827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.508737    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy\") pod \"89024f8b-846e-4309-94c8-1f0e1b05af2a\" (UID: \"89024f8b-846e-4309-94c8-1f0e1b05af2a\") "
Oct 30 20:24:30.509811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.509635    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:30.509811 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.509778    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:30.510018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.509844    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:30.510170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.510141    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:30.510240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.510222    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:30.512580 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.512544    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:30.512774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.512752    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:24:30.512878 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.512771    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "kube-api-access-rtcwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:30.513430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.513406    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy" pod "89024f8b-846e-4309-94c8-1f0e1b05af2a" (UID: "89024f8b-846e-4309-94c8-1f0e1b05af2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609654    6389 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-data\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609689    6389 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-envoy\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609700    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtcwd\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-kube-api-access-rtcwd\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609711    6389 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609720    6389 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-token\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609728    6389 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-workload-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609736    6389 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89024f8b-846e-4309-94c8-1f0e1b05af2a-istiod-ca-cert\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609745    6389 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89024f8b-846e-4309-94c8-1f0e1b05af2a-istio-podinfo\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.609743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.609754    6389 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89024f8b-846e-4309-94c8-1f0e1b05af2a-credential-socket\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:30.985146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.984357    6389 generic.go:356] "Generic (PLEG): container finished" podID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerID="60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0" exitCode=0
Oct 30 20:24:30.985146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.984454    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" event={"ID":"89024f8b-846e-4309-94c8-1f0e1b05af2a","Type":"ContainerDied","Data":"60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0"}
Oct 30 20:24:30.985146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.984510    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm" event={"ID":"89024f8b-846e-4309-94c8-1f0e1b05af2a","Type":"ContainerDied","Data":"4364a70c51dffa6582942b58c7d1431f9e218e8638bf98ae9129ffb2961200bf"}
Oct 30 20:24:30.985146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.984542    6389 scope.go:122] "RemoveContainer" containerID="60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0"
Oct 30 20:24:30.985146 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:30.984703    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"
Oct 30 20:24:31.004543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:31.004407    6389 scope.go:122] "RemoveContainer" containerID="60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0"
Oct 30 20:24:31.004752 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:31.004729    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0\": container with ID starting with 60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0 not found: ID does not exist" containerID="60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0"
Oct 30 20:24:31.004809 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:31.004763    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0"} err="failed to get container status \"60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0\": rpc error: code = NotFound desc = could not find container \"60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0\": container with ID starting with 60fe06b341a2c0ee80b45d43717854b062f817b5a20dbd9f985fe0820d2f35f0 not found: ID does not exist"
Oct 30 20:24:31.018433 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:31.018408    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"]
Oct 30 20:24:31.029727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:31.029661    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-124fd1-openshift-default-5b554bcd59-4kzbm"]
Oct 30 20:24:31.913266 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:31.913235    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" path="/var/lib/kubelet/pods/89024f8b-846e-4309-94c8-1f0e1b05af2a/volumes"
Oct 30 20:24:35.186294 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:35.186256    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:38.555373 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:38.555332    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:38.655494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:38.655460    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:39.605551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:39.605514    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:40.072575 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:40.072546    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:24:40.074993 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:40.074968    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:24:40.248699 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:40.248667    6389 scope.go:122] "RemoveContainer" containerID="f9f4f360b5c868d5fe0b21bce2cf200b094da309eff74fd6da6c9c52088c1971"
Oct 30 20:24:45.746516 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:45.746464    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:24:48.026970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:48.026935    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"]
Oct 30 20:24:48.030724 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:48.030698    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b93e785e-rbf62"]
Oct 30 20:24:49.918561 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:49.918526    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f73dc17-1f22-4d03-8745-a639b449a755" path="/var/lib/kubelet/pods/0f73dc17-1f22-4d03-8745-a639b449a755/volumes"
Oct 30 20:24:51.308754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.308714    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"]
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309140    6389 cpu_manager.go:414] "containerMap: removing container" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309152    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309174    6389 cpu_manager.go:414] "containerMap: removing container" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309181    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309188    6389 cpu_manager.go:414] "containerMap: removing container" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309192    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309202    6389 cpu_manager.go:414] "containerMap: removing container" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerName="payload-pre-processing"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309207    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerName="payload-pre-processing"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309215    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" containerName="payload-processing"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309219    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" containerName="payload-processing"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309234    6389 cpu_manager.go:414] "containerMap: removing container" podUID="719f5dee-683a-4a32-bdf6-42280a157252" containerName="revoke-keys"
Oct 30 20:24:51.309241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309239    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="719f5dee-683a-4a32-bdf6-42280a157252" containerName="revoke-keys"
Oct 30 20:24:51.309616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309311    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="831bc213-3f56-42f5-88a2-12d399c1a81e" containerName="main"
Oct 30 20:24:51.309616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309320    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="89024f8b-846e-4309-94c8-1f0e1b05af2a" containerName="istio-proxy"
Oct 30 20:24:51.309616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309332    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="4eb940e8-8ce9-4371-a732-0dcca00a0047" containerName="payload-pre-processing"
Oct 30 20:24:51.309616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309339    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40e1118-0922-4fbc-a1db-02244272ce81" containerName="payload-processing"
Oct 30 20:24:51.309616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309348    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf1c2b2c-022e-4125-8d0f-112af9b9ee5b" containerName="main"
Oct 30 20:24:51.309616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.309354    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="719f5dee-683a-4a32-bdf6-42280a157252" containerName="revoke-keys"
Oct 30 20:24:51.314250 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.314230    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.323141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.323094    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:24:51.323280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.323184    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:24:51.325400 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.325378    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"]
Oct 30 20:24:51.387368 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.387324    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.387545 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.387382    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2bn8\" (UniqueName: \"kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.387545 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.387458    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.488948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.488908    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2bn8\" (UniqueName: \"kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.489153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.489069    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.489211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.489183    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.489791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.489772    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.491448 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.491426    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.502441 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.502417    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2bn8\" (UniqueName: \"kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8\") pod \"maas-api-revoke-sub-85237e45-ndpd9\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.627855 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.627775    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:51.900640 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:51.900615    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"]
Oct 30 20:24:51.902196 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:24:51.902167    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c00cfc0_b126_47c0_9c6f_af1674603009.slice/crio-c2133d179a954e086e32748035fd0b657cd64705219c7537f6e679676a99e3b7 WatchSource:0}: Error finding container c2133d179a954e086e32748035fd0b657cd64705219c7537f6e679676a99e3b7: Status 404 returned error can't find the container with id c2133d179a954e086e32748035fd0b657cd64705219c7537f6e679676a99e3b7
Oct 30 20:24:52.115292 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:52.115254    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9" event={"ID":"7c00cfc0-b126-47c0-9c6f-af1674603009","Type":"ContainerStarted","Data":"c2133d179a954e086e32748035fd0b657cd64705219c7537f6e679676a99e3b7"}
Oct 30 20:24:53.122111 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:53.122021    6389 generic.go:356] "Generic (PLEG): container finished" podID="7c00cfc0-b126-47c0-9c6f-af1674603009" containerID="7fef7f342b799918697d48a95f1e4cc73641940831d7c62b014790fd391bce9e" exitCode=0
Oct 30 20:24:53.122561 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:53.122108    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9" event={"ID":"7c00cfc0-b126-47c0-9c6f-af1674603009","Type":"ContainerDied","Data":"7fef7f342b799918697d48a95f1e4cc73641940831d7c62b014790fd391bce9e"}
Oct 30 20:24:54.251021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.250992    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:54.310751 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.310721    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca\") pod \"7c00cfc0-b126-47c0-9c6f-af1674603009\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") "
Oct 30 20:24:54.310910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.310805    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca\") pod \"7c00cfc0-b126-47c0-9c6f-af1674603009\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") "
Oct 30 20:24:54.310992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.310913    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8\" (UniqueName: \"kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8\") pod \"7c00cfc0-b126-47c0-9c6f-af1674603009\" (UID: \"7c00cfc0-b126-47c0-9c6f-af1674603009\") "
Oct 30 20:24:54.311156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.311129    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca" pod "7c00cfc0-b126-47c0-9c6f-af1674603009" (UID: "7c00cfc0-b126-47c0-9c6f-af1674603009"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:54.312736 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.312717    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca" pod "7c00cfc0-b126-47c0-9c6f-af1674603009" (UID: "7c00cfc0-b126-47c0-9c6f-af1674603009"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:54.312876 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.312861    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8" pod "7c00cfc0-b126-47c0-9c6f-af1674603009" (UID: "7c00cfc0-b126-47c0-9c6f-af1674603009"). InnerVolumeSpecName "kube-api-access-v2bn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:54.412573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.412504    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:54.412573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.412538    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7c00cfc0-b126-47c0-9c6f-af1674603009-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:54.412573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:54.412553    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2bn8\" (UniqueName: \"kubernetes.io/projected/7c00cfc0-b126-47c0-9c6f-af1674603009-kube-api-access-v2bn8\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:55.137438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:55.137401    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9" event={"ID":"7c00cfc0-b126-47c0-9c6f-af1674603009","Type":"ContainerDied","Data":"c2133d179a954e086e32748035fd0b657cd64705219c7537f6e679676a99e3b7"}
Oct 30 20:24:55.137438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:55.137433    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"
Oct 30 20:24:55.137659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:24:55.137454    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2133d179a954e086e32748035fd0b657cd64705219c7537f6e679676a99e3b7"
Oct 30 20:24:55.791627 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:24:55.791588    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:25:02.837280 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:02.837246    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:25:04.729517 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:04.729484    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:25:05.540536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:05.540503    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:25:09.025457 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:09.025424    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"]
Oct 30 20:25:09.028951 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:09.028927    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ed2a30c7-cn6gk"]
Oct 30 20:25:09.913048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:09.913015    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc5c740b-f7bd-47a3-8b25-afc30e35f6eb" path="/var/lib/kubelet/pods/cc5c740b-f7bd-47a3-8b25-afc30e35f6eb/volumes"
Oct 30 20:25:23.027590 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:23.027556    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"]
Oct 30 20:25:23.031021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:23.030994    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e38850ba-hchfz"]
Oct 30 20:25:23.914647 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:23.914604    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9194bcce-31ad-4ce9-806e-0bb9f79866c1" path="/var/lib/kubelet/pods/9194bcce-31ad-4ce9-806e-0bb9f79866c1/volumes"
Oct 30 20:25:25.790821 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:25:25.790786    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:25:35.235071 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.235035    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:25:35.585045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.584970    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"]
Oct 30 20:25:35.585363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.585351    6389 cpu_manager.go:414] "containerMap: removing container" podUID="7c00cfc0-b126-47c0-9c6f-af1674603009" containerName="revoke-keys"
Oct 30 20:25:35.585412 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.585365    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c00cfc0-b126-47c0-9c6f-af1674603009" containerName="revoke-keys"
Oct 30 20:25:35.585460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.585449    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c00cfc0-b126-47c0-9c6f-af1674603009" containerName="revoke-keys"
Oct 30 20:25:35.588570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.588552    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.595633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.595374    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:25:35.595885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.595857    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:25:35.597674 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.597652    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"]
Oct 30 20:25:35.713575 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.713548    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.713728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.713607    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkxvw\" (UniqueName: \"kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.713728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.713711    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.814565 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.814536    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.814716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.814591    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkxvw\" (UniqueName: \"kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.814716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.814627    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.815167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.815147    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.816970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.816936    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.827395 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.827368    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkxvw\" (UniqueName: \"kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw\") pod \"maas-api-revoke-sub-c1e271a9-4cw85\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:35.904958 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:35.904908    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:36.180477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:36.180440    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"]
Oct 30 20:25:36.181925 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:25:36.181896    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5f02f4_b36f_4bdb_b882_55dc7ca90eb1.slice/crio-8584bd56ab81e8f5482caeafd03a441440a1e97a3306129f4db1ed3c28a2cc60 WatchSource:0}: Error finding container 8584bd56ab81e8f5482caeafd03a441440a1e97a3306129f4db1ed3c28a2cc60: Status 404 returned error can't find the container with id 8584bd56ab81e8f5482caeafd03a441440a1e97a3306129f4db1ed3c28a2cc60
Oct 30 20:25:36.412935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:36.412898    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85" event={"ID":"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1","Type":"ContainerStarted","Data":"8584bd56ab81e8f5482caeafd03a441440a1e97a3306129f4db1ed3c28a2cc60"}
Oct 30 20:25:37.420105 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:37.420066    6389 generic.go:356] "Generic (PLEG): container finished" podID="dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" containerID="ae23831a6b4b1bff7ca134849c113559e93c1c9f8fd71cdba5ccb6d1ab47c04a" exitCode=0
Oct 30 20:25:37.420581 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:37.420156    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85" event={"ID":"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1","Type":"ContainerDied","Data":"ae23831a6b4b1bff7ca134849c113559e93c1c9f8fd71cdba5ccb6d1ab47c04a"}
Oct 30 20:25:38.554924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.554904    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:38.743029 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.742996    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw\" (UniqueName: \"kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw\") pod \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") "
Oct 30 20:25:38.743223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.743111    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca\") pod \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") "
Oct 30 20:25:38.743223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.743183    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca\") pod \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\" (UID: \"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1\") "
Oct 30 20:25:38.743562 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.743536    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca" pod "dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" (UID: "dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:25:38.745107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.745072    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca" pod "dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" (UID: "dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:25:38.745107 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.745091    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw" pod "dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" (UID: "dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1"). InnerVolumeSpecName "kube-api-access-xkxvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:25:38.844300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.844269    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:25:38.844300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.844297    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:25:38.844469 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:38.844308    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkxvw\" (UniqueName: \"kubernetes.io/projected/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1-kube-api-access-xkxvw\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:25:39.432629 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:39.432602    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"
Oct 30 20:25:39.432785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:39.432643    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85" event={"ID":"dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1","Type":"ContainerDied","Data":"8584bd56ab81e8f5482caeafd03a441440a1e97a3306129f4db1ed3c28a2cc60"}
Oct 30 20:25:39.432785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:39.432693    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8584bd56ab81e8f5482caeafd03a441440a1e97a3306129f4db1ed3c28a2cc60"
Oct 30 20:25:42.026106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:42.026073    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"]
Oct 30 20:25:42.031992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:42.031967    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3244646-kcl4n"]
Oct 30 20:25:43.913075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:43.913042    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf297cd7-0617-437d-9c8b-32b973bf6ca9" path="/var/lib/kubelet/pods/cf297cd7-0617-437d-9c8b-32b973bf6ca9/volumes"
Oct 30 20:25:53.026164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:53.026071    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"]
Oct 30 20:25:53.029371 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:53.029343    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2ffe1807-lxbk4"]
Oct 30 20:25:53.912975 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:53.912944    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fd6c549-e68a-437c-9310-d28abca48949" path="/var/lib/kubelet/pods/2fd6c549-e68a-437c-9310-d28abca48949/volumes"
Oct 30 20:25:54.497558 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.497526    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"]
Oct 30 20:25:54.497992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.497865    6389 cpu_manager.go:414] "containerMap: removing container" podUID="dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" containerName="revoke-keys"
Oct 30 20:25:54.497992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.497875    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" containerName="revoke-keys"
Oct 30 20:25:54.497992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.497947    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" containerName="revoke-keys"
Oct 30 20:25:54.502291 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.502276    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.513411 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.513388    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:25:54.513681 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.513660    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:25:54.513931 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.513911    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"]
Oct 30 20:25:54.613586 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.613549    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.613753 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.613627    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.613753 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.613709    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xf8v\" (UniqueName: \"kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.714717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.714684    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xf8v\" (UniqueName: \"kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.714845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.714742    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.714845 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.714766    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.715428 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.715407    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.716967 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.716948    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.728427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.728404    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xf8v\" (UniqueName: \"kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v\") pod \"maas-api-revoke-sub-2e1f5b5f-g8q4n\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:54.816232 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:54.816181    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:55.090618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:55.090591    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"]
Oct 30 20:25:55.091814 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:25:55.091788    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f227e0_448c_4d42_90a5_53b7ecddd94f.slice/crio-41e068bba9ffa874c1f88aae8a5683e75d7eac8f073e74628d85eeedd3fff2df WatchSource:0}: Error finding container 41e068bba9ffa874c1f88aae8a5683e75d7eac8f073e74628d85eeedd3fff2df: Status 404 returned error can't find the container with id 41e068bba9ffa874c1f88aae8a5683e75d7eac8f073e74628d85eeedd3fff2df
Oct 30 20:25:55.532464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:55.532421    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n" event={"ID":"b2f227e0-448c-4d42-90a5-53b7ecddd94f","Type":"ContainerStarted","Data":"974b3063c2db5c8b5ba81b58a9ffecfed66447d52460cd9ff9685e5674b59bf4"}
Oct 30 20:25:55.532910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:55.532481    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n" event={"ID":"b2f227e0-448c-4d42-90a5-53b7ecddd94f","Type":"ContainerStarted","Data":"41e068bba9ffa874c1f88aae8a5683e75d7eac8f073e74628d85eeedd3fff2df"}
Oct 30 20:25:55.552960 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:55.552918    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n" podStartSLOduration=1.552903414 podStartE2EDuration="1.552903414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:25:55.551148518 +0000 UTC m=+1275.750721447" watchObservedRunningTime="2026-10-30 20:25:55.552903414 +0000 UTC m=+1275.752476343"
Oct 30 20:25:55.790508 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:25:55.790422    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:25:56.539061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:56.539025    6389 generic.go:356] "Generic (PLEG): container finished" podID="b2f227e0-448c-4d42-90a5-53b7ecddd94f" containerID="974b3063c2db5c8b5ba81b58a9ffecfed66447d52460cd9ff9685e5674b59bf4" exitCode=0
Oct 30 20:25:56.539468 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:56.539139    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n" event={"ID":"b2f227e0-448c-4d42-90a5-53b7ecddd94f","Type":"ContainerDied","Data":"974b3063c2db5c8b5ba81b58a9ffecfed66447d52460cd9ff9685e5674b59bf4"}
Oct 30 20:25:57.675759 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.675735    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:57.743828 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.743795    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v\" (UniqueName: \"kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v\") pod \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") "
Oct 30 20:25:57.743983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.743901    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca\") pod \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") "
Oct 30 20:25:57.743983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.743971    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca\") pod \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\" (UID: \"b2f227e0-448c-4d42-90a5-53b7ecddd94f\") "
Oct 30 20:25:57.744329 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.744298    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca" pod "b2f227e0-448c-4d42-90a5-53b7ecddd94f" (UID: "b2f227e0-448c-4d42-90a5-53b7ecddd94f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:25:57.745873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.745850    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca" pod "b2f227e0-448c-4d42-90a5-53b7ecddd94f" (UID: "b2f227e0-448c-4d42-90a5-53b7ecddd94f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:25:57.745945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.745899    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v" pod "b2f227e0-448c-4d42-90a5-53b7ecddd94f" (UID: "b2f227e0-448c-4d42-90a5-53b7ecddd94f"). InnerVolumeSpecName "kube-api-access-2xf8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:25:57.845256 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.845176    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:25:57.845256 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.845204    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xf8v\" (UniqueName: \"kubernetes.io/projected/b2f227e0-448c-4d42-90a5-53b7ecddd94f-kube-api-access-2xf8v\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:25:57.845256 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:57.845214    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2f227e0-448c-4d42-90a5-53b7ecddd94f-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:25:58.551472 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:58.551442    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"
Oct 30 20:25:58.551652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:58.551483    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n" event={"ID":"b2f227e0-448c-4d42-90a5-53b7ecddd94f","Type":"ContainerDied","Data":"41e068bba9ffa874c1f88aae8a5683e75d7eac8f073e74628d85eeedd3fff2df"}
Oct 30 20:25:58.551652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:25:58.551527    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41e068bba9ffa874c1f88aae8a5683e75d7eac8f073e74628d85eeedd3fff2df"
Oct 30 20:26:05.510917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.510881    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"]
Oct 30 20:26:05.511337 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.511278    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b2f227e0-448c-4d42-90a5-53b7ecddd94f" containerName="revoke-keys"
Oct 30 20:26:05.511337 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.511289    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2f227e0-448c-4d42-90a5-53b7ecddd94f" containerName="revoke-keys"
Oct 30 20:26:05.511431 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.511340    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2f227e0-448c-4d42-90a5-53b7ecddd94f" containerName="revoke-keys"
Oct 30 20:26:05.514608 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.514585    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.523036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.522479    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:26:05.523036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.522666    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:26:05.525856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.525826    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"]
Oct 30 20:26:05.623245 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.623210    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbzzv\" (UniqueName: \"kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.623452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.623263    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.623452 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.623426    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.724159 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.724107    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.724367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.724228    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.724367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.724273    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbzzv\" (UniqueName: \"kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.725055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.725026    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.726709 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.726686    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.737319 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.737258    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbzzv\" (UniqueName: \"kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv\") pod \"maas-api-revoke-sub-ec843018-x9qsm\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:05.829890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:05.829817    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:06.106692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:06.106659    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"]
Oct 30 20:26:06.108348 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:26:06.108319    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d445fd_936e_492b_bc8f_d2c2c69e7ca8.slice/crio-bb14d6a9e225a9ef266d6b5b8c06fbfb80b20751b0f07c8d8cd40212e279b05d WatchSource:0}: Error finding container bb14d6a9e225a9ef266d6b5b8c06fbfb80b20751b0f07c8d8cd40212e279b05d: Status 404 returned error can't find the container with id bb14d6a9e225a9ef266d6b5b8c06fbfb80b20751b0f07c8d8cd40212e279b05d
Oct 30 20:26:06.597494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:06.597457    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm" event={"ID":"96d445fd-936e-492b-bc8f-d2c2c69e7ca8","Type":"ContainerStarted","Data":"6e33c5cb9f683dfef3e0e549c8bdd4490718ea26af02ae5cddfca8ebb961ddb8"}
Oct 30 20:26:06.597862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:06.597545    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm" event={"ID":"96d445fd-936e-492b-bc8f-d2c2c69e7ca8","Type":"ContainerStarted","Data":"bb14d6a9e225a9ef266d6b5b8c06fbfb80b20751b0f07c8d8cd40212e279b05d"}
Oct 30 20:26:06.619403 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:06.619342    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm" podStartSLOduration=1.61932608 podStartE2EDuration="1.61932608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:26:06.617692069 +0000 UTC m=+1286.817265003" watchObservedRunningTime="2026-10-30 20:26:06.61932608 +0000 UTC m=+1286.818899009"
Oct 30 20:26:07.607622 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:07.607588    6389 generic.go:356] "Generic (PLEG): container finished" podID="96d445fd-936e-492b-bc8f-d2c2c69e7ca8" containerID="6e33c5cb9f683dfef3e0e549c8bdd4490718ea26af02ae5cddfca8ebb961ddb8" exitCode=0
Oct 30 20:26:07.608192 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:07.607679    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm" event={"ID":"96d445fd-936e-492b-bc8f-d2c2c69e7ca8","Type":"ContainerDied","Data":"6e33c5cb9f683dfef3e0e549c8bdd4490718ea26af02ae5cddfca8ebb961ddb8"}
Oct 30 20:26:08.919689 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:08.919667    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:09.088424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.088386    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv\" (UniqueName: \"kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv\") pod \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") "
Oct 30 20:26:09.088760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.088738    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca\") pod \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") "
Oct 30 20:26:09.088929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.088912    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca\") pod \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\" (UID: \"96d445fd-936e-492b-bc8f-d2c2c69e7ca8\") "
Oct 30 20:26:09.089202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.089178    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca" pod "96d445fd-936e-492b-bc8f-d2c2c69e7ca8" (UID: "96d445fd-936e-492b-bc8f-d2c2c69e7ca8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:26:09.089453 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.089435    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:09.092391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.092353    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca" pod "96d445fd-936e-492b-bc8f-d2c2c69e7ca8" (UID: "96d445fd-936e-492b-bc8f-d2c2c69e7ca8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:26:09.093111 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.093080    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv" pod "96d445fd-936e-492b-bc8f-d2c2c69e7ca8" (UID: "96d445fd-936e-492b-bc8f-d2c2c69e7ca8"). InnerVolumeSpecName "kube-api-access-nbzzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:26:09.191644 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.191546    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:09.191644 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.191597    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbzzv\" (UniqueName: \"kubernetes.io/projected/96d445fd-936e-492b-bc8f-d2c2c69e7ca8-kube-api-access-nbzzv\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:09.620041 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.620012    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"
Oct 30 20:26:09.620221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.620046    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm" event={"ID":"96d445fd-936e-492b-bc8f-d2c2c69e7ca8","Type":"ContainerDied","Data":"bb14d6a9e225a9ef266d6b5b8c06fbfb80b20751b0f07c8d8cd40212e279b05d"}
Oct 30 20:26:09.620221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:09.620104    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb14d6a9e225a9ef266d6b5b8c06fbfb80b20751b0f07c8d8cd40212e279b05d"
Oct 30 20:26:10.664577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:10.664545    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:26:11.029033 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:11.029001    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"]
Oct 30 20:26:11.034543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:11.034510    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be1f16a5-5jq96"]
Oct 30 20:26:11.912770 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:11.912736    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17cb30fb-a840-44f7-a58f-7255fb5133fe" path="/var/lib/kubelet/pods/17cb30fb-a840-44f7-a58f-7255fb5133fe/volumes"
Oct 30 20:26:24.025566 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:24.025533    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"]
Oct 30 20:26:24.028977 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:24.028947    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771980ae-h9hzd"]
Oct 30 20:26:25.791481 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:26:25.791444    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:26:25.912655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:25.912626    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9e50323-5c6e-469a-a2cc-61fb34920f58" path="/var/lib/kubelet/pods/d9e50323-5c6e-469a-a2cc-61fb34920f58/volumes"
Oct 30 20:26:26.025974 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:26.025941    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"]
Oct 30 20:26:26.029374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:26.029350    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-29346fc3-7wts2"]
Oct 30 20:26:27.913357 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:27.913321    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0f77e30-4745-4424-b7e3-76c2bbb1ea36" path="/var/lib/kubelet/pods/a0f77e30-4745-4424-b7e3-76c2bbb1ea36/volumes"
Oct 30 20:26:28.444883 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.444850    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"]
Oct 30 20:26:28.445249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.445236    6389 cpu_manager.go:414] "containerMap: removing container" podUID="96d445fd-936e-492b-bc8f-d2c2c69e7ca8" containerName="revoke-keys"
Oct 30 20:26:28.445301 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.445250    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96d445fd-936e-492b-bc8f-d2c2c69e7ca8" containerName="revoke-keys"
Oct 30 20:26:28.445341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.445331    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="96d445fd-936e-492b-bc8f-d2c2c69e7ca8" containerName="revoke-keys"
Oct 30 20:26:28.448263 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.448247    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.456312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.456287    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:26:28.456312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.456290    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:26:28.458432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.458408    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"]
Oct 30 20:26:28.579090 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.579059    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.579090 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.579093    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6kzd\" (UniqueName: \"kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.579323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.579222    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.680041 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.680008    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.680241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.680084    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.680241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.680137    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6kzd\" (UniqueName: \"kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.680768 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.680746    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.682401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.682381    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.693154 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.693106    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6kzd\" (UniqueName: \"kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd\") pod \"maas-api-revoke-sub-046e1343-j92jn\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:28.762493 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:28.762469    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:29.036398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:29.036363    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"]
Oct 30 20:26:29.038032 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:26:29.038004    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3487c011_e5f3_44fe_875d_47c2cba8d10a.slice/crio-a61bb51f41f39b6dca80fcbd11a704e84b60521db1ef9dc1da455e5a92f93d5b WatchSource:0}: Error finding container a61bb51f41f39b6dca80fcbd11a704e84b60521db1ef9dc1da455e5a92f93d5b: Status 404 returned error can't find the container with id a61bb51f41f39b6dca80fcbd11a704e84b60521db1ef9dc1da455e5a92f93d5b
Oct 30 20:26:29.755579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:29.755554    6389 generic.go:356] "Generic (PLEG): container finished" podID="3487c011-e5f3-44fe-875d-47c2cba8d10a" containerID="1679998b52ad4ff2b2be61b459c352ee8b596e2088db6cc50d1cdb297a14d56f" exitCode=0
Oct 30 20:26:29.755686 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:29.755639    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn" event={"ID":"3487c011-e5f3-44fe-875d-47c2cba8d10a","Type":"ContainerDied","Data":"1679998b52ad4ff2b2be61b459c352ee8b596e2088db6cc50d1cdb297a14d56f"}
Oct 30 20:26:29.755733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:29.755687    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn" event={"ID":"3487c011-e5f3-44fe-875d-47c2cba8d10a","Type":"ContainerStarted","Data":"a61bb51f41f39b6dca80fcbd11a704e84b60521db1ef9dc1da455e5a92f93d5b"}
Oct 30 20:26:30.893335 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:30.893309    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:31.000955 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.000922    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca\") pod \"3487c011-e5f3-44fe-875d-47c2cba8d10a\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") "
Oct 30 20:26:31.001169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.000987    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd\" (UniqueName: \"kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd\") pod \"3487c011-e5f3-44fe-875d-47c2cba8d10a\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") "
Oct 30 20:26:31.001169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.001029    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca\") pod \"3487c011-e5f3-44fe-875d-47c2cba8d10a\" (UID: \"3487c011-e5f3-44fe-875d-47c2cba8d10a\") "
Oct 30 20:26:31.001344 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.001318    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca" pod "3487c011-e5f3-44fe-875d-47c2cba8d10a" (UID: "3487c011-e5f3-44fe-875d-47c2cba8d10a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:26:31.003061 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.003027    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca" pod "3487c011-e5f3-44fe-875d-47c2cba8d10a" (UID: "3487c011-e5f3-44fe-875d-47c2cba8d10a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:26:31.003188 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.003082    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd" pod "3487c011-e5f3-44fe-875d-47c2cba8d10a" (UID: "3487c011-e5f3-44fe-875d-47c2cba8d10a"). InnerVolumeSpecName "kube-api-access-z6kzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:26:31.101841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.101757    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6kzd\" (UniqueName: \"kubernetes.io/projected/3487c011-e5f3-44fe-875d-47c2cba8d10a-kube-api-access-z6kzd\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:31.101841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.101787    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:31.101841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.101798    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3487c011-e5f3-44fe-875d-47c2cba8d10a-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:31.769077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.769046    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"
Oct 30 20:26:31.769077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.769055    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn" event={"ID":"3487c011-e5f3-44fe-875d-47c2cba8d10a","Type":"ContainerDied","Data":"a61bb51f41f39b6dca80fcbd11a704e84b60521db1ef9dc1da455e5a92f93d5b"}
Oct 30 20:26:31.769307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:31.769110    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a61bb51f41f39b6dca80fcbd11a704e84b60521db1ef9dc1da455e5a92f93d5b"
Oct 30 20:26:35.289166 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:35.289113    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:26:38.719302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.719270    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"]
Oct 30 20:26:38.719682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.719640    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3487c011-e5f3-44fe-875d-47c2cba8d10a" containerName="revoke-keys"
Oct 30 20:26:38.719682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.719651    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3487c011-e5f3-44fe-875d-47c2cba8d10a" containerName="revoke-keys"
Oct 30 20:26:38.719757 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.719711    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3487c011-e5f3-44fe-875d-47c2cba8d10a" containerName="revoke-keys"
Oct 30 20:26:38.722929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.722913    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.733730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.733686    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"]
Oct 30 20:26:38.760799 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.760771    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:26:38.761088 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.761067    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:26:38.868443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.868409    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.868616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.868476    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.868616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.868597    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s4f9\" (UniqueName: \"kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.969213 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.969179    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s4f9\" (UniqueName: \"kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.969414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.969225    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.969414 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.969278    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.969969 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.969939    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.971806 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.971786    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:38.981933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:38.981909    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s4f9\" (UniqueName: \"kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9\") pod \"maas-api-revoke-sub-d3fba056-xc477\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:39.037903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:39.037877    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:39.314046 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:39.314020    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"]
Oct 30 20:26:39.315364 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:26:39.315329    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a711fe_57d4_4485_9a83_453f4b7c434f.slice/crio-c515cdea07f595ac88d786db4266023f5ecb34010d52b1b4ee6ead1096785035 WatchSource:0}: Error finding container c515cdea07f595ac88d786db4266023f5ecb34010d52b1b4ee6ead1096785035: Status 404 returned error can't find the container with id c515cdea07f595ac88d786db4266023f5ecb34010d52b1b4ee6ead1096785035
Oct 30 20:26:39.818725 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:39.818677    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477" event={"ID":"b8a711fe-57d4-4485-9a83-453f4b7c434f","Type":"ContainerStarted","Data":"04e1739553a8b1d218208b9db6d0f13c86188e0536fc0caacd8e915779789e62"}
Oct 30 20:26:39.818725 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:39.818733    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477" event={"ID":"b8a711fe-57d4-4485-9a83-453f4b7c434f","Type":"ContainerStarted","Data":"c515cdea07f595ac88d786db4266023f5ecb34010d52b1b4ee6ead1096785035"}
Oct 30 20:26:39.839519 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:39.839456    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477" podStartSLOduration=1.839406768 podStartE2EDuration="1.839406768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:26:39.83784275 +0000 UTC m=+1320.037415679" watchObservedRunningTime="2026-10-30 20:26:39.839406768 +0000 UTC m=+1320.038979689"
Oct 30 20:26:40.322957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:40.322924    6389 scope.go:122] "RemoveContainer" containerID="611128eb9aff2b27364b9142657c85391bad46f7c0a91abf6b678f21965216a3"
Oct 30 20:26:40.334196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:40.334172    6389 scope.go:122] "RemoveContainer" containerID="a7380fb58fc122387e38da2aab4b6b4b490f1965631fc65cc4fe1b032826defe"
Oct 30 20:26:40.342865 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:40.342762    6389 scope.go:122] "RemoveContainer" containerID="eb3c035cf439295a3121597241e8d0f57a28d9897b52055d3ca7e8672d67c883"
Oct 30 20:26:40.351724 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:40.351701    6389 scope.go:122] "RemoveContainer" containerID="9c4e18918ff2455577640303f20b572d3c2dbd7a2f6dafa76a8a7f78920bf42c"
Oct 30 20:26:40.860196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:40.859652    6389 generic.go:356] "Generic (PLEG): container finished" podID="b8a711fe-57d4-4485-9a83-453f4b7c434f" containerID="04e1739553a8b1d218208b9db6d0f13c86188e0536fc0caacd8e915779789e62" exitCode=0
Oct 30 20:26:40.860196 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:40.859909    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477" event={"ID":"b8a711fe-57d4-4485-9a83-453f4b7c434f","Type":"ContainerDied","Data":"04e1739553a8b1d218208b9db6d0f13c86188e0536fc0caacd8e915779789e62"}
Oct 30 20:26:41.074143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:41.073176    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"]
Oct 30 20:26:41.092219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:41.092179    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03eedbe0-l282n"]
Oct 30 20:26:41.914977 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:41.914941    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8108047e-dc3f-4b7c-80a8-78ad3dcec743" path="/var/lib/kubelet/pods/8108047e-dc3f-4b7c-80a8-78ad3dcec743/volumes"
Oct 30 20:26:42.005533 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.005498    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:26:42.005698 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.005560    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:26:42.072425 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.072397    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:42.109446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.109403    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca\") pod \"b8a711fe-57d4-4485-9a83-453f4b7c434f\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") "
Oct 30 20:26:42.109623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.109535    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca\") pod \"b8a711fe-57d4-4485-9a83-453f4b7c434f\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") "
Oct 30 20:26:42.109623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.109613    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9\" (UniqueName: \"kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9\") pod \"b8a711fe-57d4-4485-9a83-453f4b7c434f\" (UID: \"b8a711fe-57d4-4485-9a83-453f4b7c434f\") "
Oct 30 20:26:42.109946 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.109912    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca" pod "b8a711fe-57d4-4485-9a83-453f4b7c434f" (UID: "b8a711fe-57d4-4485-9a83-453f4b7c434f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:26:42.111943 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.111911    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9" pod "b8a711fe-57d4-4485-9a83-453f4b7c434f" (UID: "b8a711fe-57d4-4485-9a83-453f4b7c434f"). InnerVolumeSpecName "kube-api-access-2s4f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:26:42.112055 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.111949    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca" pod "b8a711fe-57d4-4485-9a83-453f4b7c434f" (UID: "b8a711fe-57d4-4485-9a83-453f4b7c434f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:26:42.210454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.210365    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:42.210454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.210405    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s4f9\" (UniqueName: \"kubernetes.io/projected/b8a711fe-57d4-4485-9a83-453f4b7c434f-kube-api-access-2s4f9\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:42.210454 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.210415    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8a711fe-57d4-4485-9a83-453f4b7c434f-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:26:42.886626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.886595    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"
Oct 30 20:26:42.886810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.886596    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477" event={"ID":"b8a711fe-57d4-4485-9a83-453f4b7c434f","Type":"ContainerDied","Data":"c515cdea07f595ac88d786db4266023f5ecb34010d52b1b4ee6ead1096785035"}
Oct 30 20:26:42.886810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:42.886727    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c515cdea07f595ac88d786db4266023f5ecb34010d52b1b4ee6ead1096785035"
Oct 30 20:26:45.028463 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:45.028427    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"]
Oct 30 20:26:45.032528 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:45.032498    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1244749a-552gz"]
Oct 30 20:26:45.912702 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:45.912660    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ca1a0c2-8981-4feb-b671-d7e86fc62070" path="/var/lib/kubelet/pods/3ca1a0c2-8981-4feb-b671-d7e86fc62070/volumes"
Oct 30 20:26:51.912323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:51.912291    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:26:55.791089 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:26:55.791052    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:26:58.024074 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:58.024043    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"]
Oct 30 20:26:58.025987 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:58.025966    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd41f1f2-9gt9m"]
Oct 30 20:26:59.614922 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.614891    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"]
Oct 30 20:26:59.615510 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.615488    6389 cpu_manager.go:414] "containerMap: removing container" podUID="b8a711fe-57d4-4485-9a83-453f4b7c434f" containerName="revoke-keys"
Oct 30 20:26:59.615606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.615511    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8a711fe-57d4-4485-9a83-453f4b7c434f" containerName="revoke-keys"
Oct 30 20:26:59.615678 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.615666    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8a711fe-57d4-4485-9a83-453f4b7c434f" containerName="revoke-keys"
Oct 30 20:26:59.618749 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.618729    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.626707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.626665    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:26:59.626827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.626733    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:26:59.628323 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.628294    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"]
Oct 30 20:26:59.667989 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.667957    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rlvc\" (UniqueName: \"kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.668132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.667999    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.668132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.668073    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.768676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.768641    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rlvc\" (UniqueName: \"kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.768676 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.768681    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.768872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.768721    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.769364 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.769343    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.771021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.771000    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.781429 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.781402    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rlvc\" (UniqueName: \"kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc\") pod \"maas-api-revoke-sub-ae594979-glntk\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:26:59.913209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.913111    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21e66127-3636-4dc8-b8ec-2749c9265124" path="/var/lib/kubelet/pods/21e66127-3636-4dc8-b8ec-2749c9265124/volumes"
Oct 30 20:26:59.933034 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:26:59.933009    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:27:00.215639 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:00.215612    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"]
Oct 30 20:27:00.217268 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:27:00.217234    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db0090f_38cc_4d15_a579_bda480337bea.slice/crio-e81138bafe227e3785edd8b4afa40da53eb46c5a31a761b8bb500014d2336294 WatchSource:0}: Error finding container e81138bafe227e3785edd8b4afa40da53eb46c5a31a761b8bb500014d2336294: Status 404 returned error can't find the container with id e81138bafe227e3785edd8b4afa40da53eb46c5a31a761b8bb500014d2336294
Oct 30 20:27:01.024971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:01.024935    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk" event={"ID":"4db0090f-38cc-4d15-a579-bda480337bea","Type":"ContainerStarted","Data":"46641a540cf10fba4ee900fb724e96e3f129ad7e25079a3f09cb09afad13c423"}
Oct 30 20:27:01.025406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:01.025001    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk" event={"ID":"4db0090f-38cc-4d15-a579-bda480337bea","Type":"ContainerStarted","Data":"e81138bafe227e3785edd8b4afa40da53eb46c5a31a761b8bb500014d2336294"}
Oct 30 20:27:01.044494 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:01.044446    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk" podStartSLOduration=2.044430862 podStartE2EDuration="2.044430862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:26:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:27:01.043612866 +0000 UTC m=+1341.243185795" watchObservedRunningTime="2026-10-30 20:27:01.044430862 +0000 UTC m=+1341.244003790"
Oct 30 20:27:02.033238 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:02.033199    6389 generic.go:356] "Generic (PLEG): container finished" podID="4db0090f-38cc-4d15-a579-bda480337bea" containerID="46641a540cf10fba4ee900fb724e96e3f129ad7e25079a3f09cb09afad13c423" exitCode=0
Oct 30 20:27:02.033626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:02.033283    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk" event={"ID":"4db0090f-38cc-4d15-a579-bda480337bea","Type":"ContainerDied","Data":"46641a540cf10fba4ee900fb724e96e3f129ad7e25079a3f09cb09afad13c423"}
Oct 30 20:27:03.167649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.167626    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:27:03.299791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.299712    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca\") pod \"4db0090f-38cc-4d15-a579-bda480337bea\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") "
Oct 30 20:27:03.299791 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.299762    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc\" (UniqueName: \"kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc\") pod \"4db0090f-38cc-4d15-a579-bda480337bea\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") "
Oct 30 20:27:03.300002 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.299836    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca\") pod \"4db0090f-38cc-4d15-a579-bda480337bea\" (UID: \"4db0090f-38cc-4d15-a579-bda480337bea\") "
Oct 30 20:27:03.300228 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.300203    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca" pod "4db0090f-38cc-4d15-a579-bda480337bea" (UID: "4db0090f-38cc-4d15-a579-bda480337bea"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:27:03.301726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.301702    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca" pod "4db0090f-38cc-4d15-a579-bda480337bea" (UID: "4db0090f-38cc-4d15-a579-bda480337bea"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:27:03.301849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.301829    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc" pod "4db0090f-38cc-4d15-a579-bda480337bea" (UID: "4db0090f-38cc-4d15-a579-bda480337bea"). InnerVolumeSpecName "kube-api-access-6rlvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:27:03.401397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.401368    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:03.401397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.401394    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4db0090f-38cc-4d15-a579-bda480337bea-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:03.401397 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:03.401404    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rlvc\" (UniqueName: \"kubernetes.io/projected/4db0090f-38cc-4d15-a579-bda480337bea-kube-api-access-6rlvc\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:04.046330 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:04.046301    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"
Oct 30 20:27:04.046525 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:04.046333    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk" event={"ID":"4db0090f-38cc-4d15-a579-bda480337bea","Type":"ContainerDied","Data":"e81138bafe227e3785edd8b4afa40da53eb46c5a31a761b8bb500014d2336294"}
Oct 30 20:27:04.046525 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:04.046403    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e81138bafe227e3785edd8b4afa40da53eb46c5a31a761b8bb500014d2336294"
Oct 30 20:27:05.026142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:05.026097    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"]
Oct 30 20:27:05.029127 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:05.029086    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-93041bbd-gmw5b"]
Oct 30 20:27:05.912599 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:05.912566    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117" path="/var/lib/kubelet/pods/f6a3fb13-6e0b-4d33-ba42-73dc5f8d9117/volumes"
Oct 30 20:27:14.839141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:14.837246    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-ndkhp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:27:14.839141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:14.837314    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-ndkhp" podUID="3323ebee-b8a0-47a1-94d2-d04a8bed6ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:27:17.884511 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.884475    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"]
Oct 30 20:27:17.884959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.884870    6389 cpu_manager.go:414] "containerMap: removing container" podUID="4db0090f-38cc-4d15-a579-bda480337bea" containerName="revoke-keys"
Oct 30 20:27:17.884959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.884882    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4db0090f-38cc-4d15-a579-bda480337bea" containerName="revoke-keys"
Oct 30 20:27:17.884959 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.884960    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="4db0090f-38cc-4d15-a579-bda480337bea" containerName="revoke-keys"
Oct 30 20:27:17.889225 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.889207    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:17.897184 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.897158    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:27:17.897292 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.897182    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:27:17.899788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.899766    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"]
Oct 30 20:27:17.938274 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.938243    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:17.938408 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.938305    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mgnd\" (UniqueName: \"kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:17.938449 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:17.938411    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.039017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.038985    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mgnd\" (UniqueName: \"kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.039195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.039046    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.039195 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.039099    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.039798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.039778    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.041443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.041421    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.052458 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.052432    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mgnd\" (UniqueName: \"kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd\") pod \"maas-api-revoke-sub-a4aef7d7-lpqv6\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.203036 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.202956    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:18.493641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:18.493614    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"]
Oct 30 20:27:18.495906 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:27:18.495880    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27e5a87_43d3_4eed_af56_1b832dcedb31.slice/crio-35b1347f49412b77bde9c31cea93019954f4f7278cfb3abb46d3cb58d014a8fa WatchSource:0}: Error finding container 35b1347f49412b77bde9c31cea93019954f4f7278cfb3abb46d3cb58d014a8fa: Status 404 returned error can't find the container with id 35b1347f49412b77bde9c31cea93019954f4f7278cfb3abb46d3cb58d014a8fa
Oct 30 20:27:19.137543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:19.137454    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6" event={"ID":"f27e5a87-43d3-4eed-af56-1b832dcedb31","Type":"ContainerStarted","Data":"363a0822224636f37d04fde1bf410d999d706cf2e3b5547bc08355d592212143"}
Oct 30 20:27:19.137543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:19.137524    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6" event={"ID":"f27e5a87-43d3-4eed-af56-1b832dcedb31","Type":"ContainerStarted","Data":"35b1347f49412b77bde9c31cea93019954f4f7278cfb3abb46d3cb58d014a8fa"}
Oct 30 20:27:19.159785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:19.159734    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6" podStartSLOduration=2.159720446 podStartE2EDuration="2.159720446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:27:19.158716136 +0000 UTC m=+1359.358289065" watchObservedRunningTime="2026-10-30 20:27:19.159720446 +0000 UTC m=+1359.359293373"
Oct 30 20:27:20.144968 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:20.144932    6389 generic.go:356] "Generic (PLEG): container finished" podID="f27e5a87-43d3-4eed-af56-1b832dcedb31" containerID="363a0822224636f37d04fde1bf410d999d706cf2e3b5547bc08355d592212143" exitCode=0
Oct 30 20:27:20.145536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:20.144986    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6" event={"ID":"f27e5a87-43d3-4eed-af56-1b832dcedb31","Type":"ContainerDied","Data":"363a0822224636f37d04fde1bf410d999d706cf2e3b5547bc08355d592212143"}
Oct 30 20:27:21.291731 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.291702    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:21.365076 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.365047    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca\") pod \"f27e5a87-43d3-4eed-af56-1b832dcedb31\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") "
Oct 30 20:27:21.365249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.365095    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca\") pod \"f27e5a87-43d3-4eed-af56-1b832dcedb31\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") "
Oct 30 20:27:21.365249 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.365159    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd\" (UniqueName: \"kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd\") pod \"f27e5a87-43d3-4eed-af56-1b832dcedb31\" (UID: \"f27e5a87-43d3-4eed-af56-1b832dcedb31\") "
Oct 30 20:27:21.365477 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.365451    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca" pod "f27e5a87-43d3-4eed-af56-1b832dcedb31" (UID: "f27e5a87-43d3-4eed-af56-1b832dcedb31"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:27:21.367211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.367171    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca" pod "f27e5a87-43d3-4eed-af56-1b832dcedb31" (UID: "f27e5a87-43d3-4eed-af56-1b832dcedb31"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:27:21.367356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.367232    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd" pod "f27e5a87-43d3-4eed-af56-1b832dcedb31" (UID: "f27e5a87-43d3-4eed-af56-1b832dcedb31"). InnerVolumeSpecName "kube-api-access-8mgnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:27:21.466434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.466410    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:21.466434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.466435    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f27e5a87-43d3-4eed-af56-1b832dcedb31-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:21.466626 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:21.466446    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mgnd\" (UniqueName: \"kubernetes.io/projected/f27e5a87-43d3-4eed-af56-1b832dcedb31-kube-api-access-8mgnd\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:22.157936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:22.157900    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"
Oct 30 20:27:22.157936 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:22.157923    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6" event={"ID":"f27e5a87-43d3-4eed-af56-1b832dcedb31","Type":"ContainerDied","Data":"35b1347f49412b77bde9c31cea93019954f4f7278cfb3abb46d3cb58d014a8fa"}
Oct 30 20:27:22.158175 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:22.157982    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35b1347f49412b77bde9c31cea93019954f4f7278cfb3abb46d3cb58d014a8fa"
Oct 30 20:27:25.790812 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:27:25.790767    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:27:35.488456 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.488412    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:27:35.964198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.963285    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"]
Oct 30 20:27:35.964198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.963890    6389 cpu_manager.go:414] "containerMap: removing container" podUID="f27e5a87-43d3-4eed-af56-1b832dcedb31" containerName="revoke-keys"
Oct 30 20:27:35.964198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.963910    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f27e5a87-43d3-4eed-af56-1b832dcedb31" containerName="revoke-keys"
Oct 30 20:27:35.964198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.964019    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="f27e5a87-43d3-4eed-af56-1b832dcedb31" containerName="revoke-keys"
Oct 30 20:27:35.969137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.967781    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:35.969137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.968248    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"]
Oct 30 20:27:35.981521 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.979368    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:27:35.981521 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:35.979472    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:27:36.106917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.106877    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q99pt\" (UniqueName: \"kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.107144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.106972    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.107144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.107042    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.207945 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.207894    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.208155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.208040    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q99pt\" (UniqueName: \"kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.208155 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.208100    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.208599 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.208576    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.210361 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.210341    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.225297 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.225272    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q99pt\" (UniqueName: \"kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt\") pod \"maas-api-revoke-sub-79805ca9-qc2qk\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.287754 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.287735    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:36.565550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:36.565519    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"]
Oct 30 20:27:36.567032 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:27:36.567004    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65093664_23bd_476e_896e_2b04c8ebd620.slice/crio-e4090296d080c83d686a3f9f394939247d3c3959e49c4d9b34932b3d8c16dba8 WatchSource:0}: Error finding container e4090296d080c83d686a3f9f394939247d3c3959e49c4d9b34932b3d8c16dba8: Status 404 returned error can't find the container with id e4090296d080c83d686a3f9f394939247d3c3959e49c4d9b34932b3d8c16dba8
Oct 30 20:27:37.284227 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:37.284193    6389 generic.go:356] "Generic (PLEG): container finished" podID="65093664-23bd-476e-896e-2b04c8ebd620" containerID="37c5f68b972c212db01670a713ffd7273304ce295ff855db3c947e3032215c53" exitCode=0
Oct 30 20:27:37.284377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:37.284281    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk" event={"ID":"65093664-23bd-476e-896e-2b04c8ebd620","Type":"ContainerDied","Data":"37c5f68b972c212db01670a713ffd7273304ce295ff855db3c947e3032215c53"}
Oct 30 20:27:37.284377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:37.284335    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk" event={"ID":"65093664-23bd-476e-896e-2b04c8ebd620","Type":"ContainerStarted","Data":"e4090296d080c83d686a3f9f394939247d3c3959e49c4d9b34932b3d8c16dba8"}
Oct 30 20:27:38.027529 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.027497    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"]
Oct 30 20:27:38.030850 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.030825    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-01e97a29-jnppp"]
Oct 30 20:27:38.422849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.422823    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:38.529973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.529934    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt\" (UniqueName: \"kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt\") pod \"65093664-23bd-476e-896e-2b04c8ebd620\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") "
Oct 30 20:27:38.530208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.530011    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca\") pod \"65093664-23bd-476e-896e-2b04c8ebd620\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") "
Oct 30 20:27:38.530208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.530054    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca\") pod \"65093664-23bd-476e-896e-2b04c8ebd620\" (UID: \"65093664-23bd-476e-896e-2b04c8ebd620\") "
Oct 30 20:27:38.530421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.530396    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca" pod "65093664-23bd-476e-896e-2b04c8ebd620" (UID: "65093664-23bd-476e-896e-2b04c8ebd620"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:27:38.532095 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.532074    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca" pod "65093664-23bd-476e-896e-2b04c8ebd620" (UID: "65093664-23bd-476e-896e-2b04c8ebd620"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:27:38.532191 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.532074    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt" pod "65093664-23bd-476e-896e-2b04c8ebd620" (UID: "65093664-23bd-476e-896e-2b04c8ebd620"). InnerVolumeSpecName "kube-api-access-q99pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:27:38.631839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.631756    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q99pt\" (UniqueName: \"kubernetes.io/projected/65093664-23bd-476e-896e-2b04c8ebd620-kube-api-access-q99pt\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:38.631839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.631790    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:38.631839 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:38.631801    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65093664-23bd-476e-896e-2b04c8ebd620-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:39.303827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:39.303795    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"
Oct 30 20:27:39.303827 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:39.303816    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk" event={"ID":"65093664-23bd-476e-896e-2b04c8ebd620","Type":"ContainerDied","Data":"e4090296d080c83d686a3f9f394939247d3c3959e49c4d9b34932b3d8c16dba8"}
Oct 30 20:27:39.304288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:39.303873    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4090296d080c83d686a3f9f394939247d3c3959e49c4d9b34932b3d8c16dba8"
Oct 30 20:27:39.913853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:39.913822    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e30a0cb-5000-48c6-9a9a-f53b2a250d9b" path="/var/lib/kubelet/pods/8e30a0cb-5000-48c6-9a9a-f53b2a250d9b/volumes"
Oct 30 20:27:40.520983 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:40.520950    6389 scope.go:122] "RemoveContainer" containerID="49763cdc5464c64c38a7a3faaeb5c1cbe01eea9e277b3c7a71bd2dc94fc85223"
Oct 30 20:27:40.529927 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:40.529908    6389 scope.go:122] "RemoveContainer" containerID="79d22941155679f167126959f9711d0796ca41533ce7daa5d71772c95884a3cd"
Oct 30 20:27:40.537710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:40.537690    6389 scope.go:122] "RemoveContainer" containerID="0db957b49549c2e5a18865e497714a876a346f6f25ae24483d454e6bff234f09"
Oct 30 20:27:40.545459 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:40.545431    6389 scope.go:122] "RemoveContainer" containerID="b2c285949b1420970d443d480af89e1bc8e087f4ba47740c16ebcfda5da7358d"
Oct 30 20:27:40.553406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:40.553389    6389 scope.go:122] "RemoveContainer" containerID="30694bf0fb8f1b3d51dd46d82318425caed5495c3e65a285ebb26c53a147abbf"
Oct 30 20:27:46.032147 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:46.032090    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"]
Oct 30 20:27:46.035766 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:46.035737    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc94a4fb-ndx7r"]
Oct 30 20:27:47.913182 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:47.913152    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf" path="/var/lib/kubelet/pods/95a73c0a-3dfb-4bdc-8c4a-3489b4273ccf/volumes"
Oct 30 20:27:54.084402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.084373    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"]
Oct 30 20:27:54.084801 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.084725    6389 cpu_manager.go:414] "containerMap: removing container" podUID="65093664-23bd-476e-896e-2b04c8ebd620" containerName="revoke-keys"
Oct 30 20:27:54.084801 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.084735    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65093664-23bd-476e-896e-2b04c8ebd620" containerName="revoke-keys"
Oct 30 20:27:54.084917 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.084867    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="65093664-23bd-476e-896e-2b04c8ebd620" containerName="revoke-keys"
Oct 30 20:27:54.090001 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.089983    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.095786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.095734    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:27:54.095948 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.095759    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:27:54.098093 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.098072    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"]
Oct 30 20:27:54.193130 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.193093    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.193273 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.193187    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.193312 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.193280    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwbwz\" (UniqueName: \"kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.294421 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.294391    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwbwz\" (UniqueName: \"kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.294576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.294430    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.294576 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.294453    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.295045 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.295027    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.296582 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.296558    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.307075 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.307053    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwbwz\" (UniqueName: \"kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz\") pod \"maas-api-revoke-sub-0b7e8edb-qd46n\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.405252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.405202    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:54.684627 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:54.684603    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"]
Oct 30 20:27:54.686420 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:27:54.686394    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3e19c9_e5a8_4ba6_859c_bdc35457747a.slice/crio-295076ec7bc73ae0ae2d1fdd69210a010fe869e43a277152ec49fba2bb562701 WatchSource:0}: Error finding container 295076ec7bc73ae0ae2d1fdd69210a010fe869e43a277152ec49fba2bb562701: Status 404 returned error can't find the container with id 295076ec7bc73ae0ae2d1fdd69210a010fe869e43a277152ec49fba2bb562701
Oct 30 20:27:55.422760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:55.422728    6389 generic.go:356] "Generic (PLEG): container finished" podID="4d3e19c9-e5a8-4ba6-859c-bdc35457747a" containerID="1ee3e045dd5fa9213ae32aaf03c30170f8c04071b6b7677cefc8c3a8a9503b5b" exitCode=0
Oct 30 20:27:55.423100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:55.422775    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n" event={"ID":"4d3e19c9-e5a8-4ba6-859c-bdc35457747a","Type":"ContainerDied","Data":"1ee3e045dd5fa9213ae32aaf03c30170f8c04071b6b7677cefc8c3a8a9503b5b"}
Oct 30 20:27:55.423100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:55.422816    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n" event={"ID":"4d3e19c9-e5a8-4ba6-859c-bdc35457747a","Type":"ContainerStarted","Data":"295076ec7bc73ae0ae2d1fdd69210a010fe869e43a277152ec49fba2bb562701"}
Oct 30 20:27:55.790577 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:27:55.790542    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:27:56.553547 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.553520    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:56.614432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.614401    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz\" (UniqueName: \"kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz\") pod \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") "
Oct 30 20:27:56.614587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.614453    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca\") pod \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") "
Oct 30 20:27:56.614650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.614596    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca\") pod \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\" (UID: \"4d3e19c9-e5a8-4ba6-859c-bdc35457747a\") "
Oct 30 20:27:56.614921 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.614896    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca" pod "4d3e19c9-e5a8-4ba6-859c-bdc35457747a" (UID: "4d3e19c9-e5a8-4ba6-859c-bdc35457747a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:27:56.616460 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.616441    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz" pod "4d3e19c9-e5a8-4ba6-859c-bdc35457747a" (UID: "4d3e19c9-e5a8-4ba6-859c-bdc35457747a"). InnerVolumeSpecName "kube-api-access-cwbwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:27:56.616543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.616463    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca" pod "4d3e19c9-e5a8-4ba6-859c-bdc35457747a" (UID: "4d3e19c9-e5a8-4ba6-859c-bdc35457747a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:27:56.715209 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.715188    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:56.715300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.715210    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwbwz\" (UniqueName: \"kubernetes.io/projected/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-kube-api-access-cwbwz\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:56.715300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:56.715221    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d3e19c9-e5a8-4ba6-859c-bdc35457747a-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:27:57.435570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:57.435541    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"
Oct 30 20:27:57.435743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:57.435535    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n" event={"ID":"4d3e19c9-e5a8-4ba6-859c-bdc35457747a","Type":"ContainerDied","Data":"295076ec7bc73ae0ae2d1fdd69210a010fe869e43a277152ec49fba2bb562701"}
Oct 30 20:27:57.435743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:27:57.435674    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="295076ec7bc73ae0ae2d1fdd69210a010fe869e43a277152ec49fba2bb562701"
Oct 30 20:28:04.804028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.803995    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"]
Oct 30 20:28:04.804406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.804365    6389 cpu_manager.go:414] "containerMap: removing container" podUID="4d3e19c9-e5a8-4ba6-859c-bdc35457747a" containerName="revoke-keys"
Oct 30 20:28:04.804406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.804376    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d3e19c9-e5a8-4ba6-859c-bdc35457747a" containerName="revoke-keys"
Oct 30 20:28:04.804479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.804444    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d3e19c9-e5a8-4ba6-859c-bdc35457747a" containerName="revoke-keys"
Oct 30 20:28:04.807543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.807526    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.817636 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.817606    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"]
Oct 30 20:28:04.868082 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.868055    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:28:04.868082 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.868079    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:28:04.884591 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.884567    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t8jg\" (UniqueName: \"kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.884728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.884615    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.884728 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.884709    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.985449 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.985421    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t8jg\" (UniqueName: \"kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.985581 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.985482    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.985581 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.985534    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.986140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.986096    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.987851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.987831    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:04.998094 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:04.998074    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t8jg\" (UniqueName: \"kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg\") pod \"maas-api-revoke-sub-3d4156f1-b7tn8\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:05.122197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:05.122139    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:05.401103 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:05.401069    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"]
Oct 30 20:28:05.402009 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:28:05.401982    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1888692_4ce9_4e60_a913_0197d50e2084.slice/crio-240d24c06966128fc62ef64f2c7f8a6511d2730d6e75f4784fd8cffa3c24eafe WatchSource:0}: Error finding container 240d24c06966128fc62ef64f2c7f8a6511d2730d6e75f4784fd8cffa3c24eafe: Status 404 returned error can't find the container with id 240d24c06966128fc62ef64f2c7f8a6511d2730d6e75f4784fd8cffa3c24eafe
Oct 30 20:28:05.481727 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:05.481697    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8" event={"ID":"f1888692-4ce9-4e60-a913-0197d50e2084","Type":"ContainerStarted","Data":"240d24c06966128fc62ef64f2c7f8a6511d2730d6e75f4784fd8cffa3c24eafe"}
Oct 30 20:28:06.488068 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:06.488034    6389 generic.go:356] "Generic (PLEG): container finished" podID="f1888692-4ce9-4e60-a913-0197d50e2084" containerID="cad60c6d33dfe9aa5be82d4badc1ef8a28bf4b12dc23fa9c43b70dc8f4a92c16" exitCode=0
Oct 30 20:28:06.488536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:06.488134    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8" event={"ID":"f1888692-4ce9-4e60-a913-0197d50e2084","Type":"ContainerDied","Data":"cad60c6d33dfe9aa5be82d4badc1ef8a28bf4b12dc23fa9c43b70dc8f4a92c16"}
Oct 30 20:28:07.622058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.622035    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:07.709215 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.709182    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg\" (UniqueName: \"kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg\") pod \"f1888692-4ce9-4e60-a913-0197d50e2084\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") "
Oct 30 20:28:07.709401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.709288    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca\") pod \"f1888692-4ce9-4e60-a913-0197d50e2084\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") "
Oct 30 20:28:07.709401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.709339    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca\") pod \"f1888692-4ce9-4e60-a913-0197d50e2084\" (UID: \"f1888692-4ce9-4e60-a913-0197d50e2084\") "
Oct 30 20:28:07.709703 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.709677    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca" pod "f1888692-4ce9-4e60-a913-0197d50e2084" (UID: "f1888692-4ce9-4e60-a913-0197d50e2084"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:28:07.711281 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.711262    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca" pod "f1888692-4ce9-4e60-a913-0197d50e2084" (UID: "f1888692-4ce9-4e60-a913-0197d50e2084"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:28:07.711383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.711364    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg" pod "f1888692-4ce9-4e60-a913-0197d50e2084" (UID: "f1888692-4ce9-4e60-a913-0197d50e2084"). InnerVolumeSpecName "kube-api-access-7t8jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:28:07.811113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.811002    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-xks-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:28:07.811113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.811051    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f1888692-4ce9-4e60-a913-0197d50e2084-maas-api-openshift-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:28:07.811113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:07.811068    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t8jg\" (UniqueName: \"kubernetes.io/projected/f1888692-4ce9-4e60-a913-0197d50e2084-kube-api-access-7t8jg\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:28:08.499741 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:08.499704    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8" event={"ID":"f1888692-4ce9-4e60-a913-0197d50e2084","Type":"ContainerDied","Data":"240d24c06966128fc62ef64f2c7f8a6511d2730d6e75f4784fd8cffa3c24eafe"}
Oct 30 20:28:08.499741 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:08.499730    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"
Oct 30 20:28:08.499958 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:08.499778    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="240d24c06966128fc62ef64f2c7f8a6511d2730d6e75f4784fd8cffa3c24eafe"
Oct 30 20:28:22.029393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:22.029356    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"]
Oct 30 20:28:22.032417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:22.032389    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-609ba8d0-2z7lm"]
Oct 30 20:28:22.886775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:22.886732    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"]
Oct 30 20:28:22.892150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:22.892106    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b7tn8"]
Oct 30 20:28:23.913331 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:23.913295    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7463ae95-2cbb-4615-abd0-eba3df86c4b1" path="/var/lib/kubelet/pods/7463ae95-2cbb-4615-abd0-eba3df86c4b1/volumes"
Oct 30 20:28:23.913709 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:23.913678    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1888692-4ce9-4e60-a913-0197d50e2084" path="/var/lib/kubelet/pods/f1888692-4ce9-4e60-a913-0197d50e2084/volumes"
Oct 30 20:28:25.790644 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:28:25.790610    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:28:35.591430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:35.591383    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:28:40.593239 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:40.593208    6389 scope.go:122] "RemoveContainer" containerID="6f4484fa484c8f5e01bbddbcc01e0b071f97bdfa72de45dd92a58b28916424df"
Oct 30 20:28:40.603671 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:40.603647    6389 scope.go:122] "RemoveContainer" containerID="9be3ab59e3a18b98aa8088067b795c0be5f5ff718906f563d18fdd8f0d347e56"
Oct 30 20:28:40.613630 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:40.613607    6389 scope.go:122] "RemoveContainer" containerID="2e7a9746f09229ed9fd906f8ca4ef25e8ff314745e4f890289e361c6225b24bf"
Oct 30 20:28:40.622048 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:40.622028    6389 scope.go:122] "RemoveContainer" containerID="b1ff398d2c3ed53b507cec23b94d96423ba6b391c992b5ef39eb47849be2be0a"
Oct 30 20:28:40.630520 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:40.630497    6389 scope.go:122] "RemoveContainer" containerID="924136df4cc59e54b1eb26a4d413bab641381e5557e0daf1064f483f8180e3db"
Oct 30 20:28:40.639253 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:28:40.639230    6389 scope.go:122] "RemoveContainer" containerID="19ddb4f40f053f04d7fbe6da7defc2a881927a6ce81aac97128dc94bad937bc5"
Oct 30 20:28:55.790851 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:28:55.790751    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:29:03.081211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:03.081169    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:03.176956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:03.176923    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:05.027896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:05.027862    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"]
Oct 30 20:29:05.032156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:05.032132    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73dfb91d-5jrtq"]
Oct 30 20:29:05.579371 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:05.579338    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:05.769234 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:05.769196    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:05.912267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:05.912183    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="719f5dee-683a-4a32-bdf6-42280a157252" path="/var/lib/kubelet/pods/719f5dee-683a-4a32-bdf6-42280a157252/volumes"
Oct 30 20:29:06.780508 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:06.780478    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:06.893164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:06.893104    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:06.998788 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:06.998749    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:07.107438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:07.107358    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:07.860193 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:07.860156    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:08.601861 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:08.601830    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:08.887985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:08.887895    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:09.496924 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:09.496889    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:10.266976 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:10.266942    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:11.089872 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:11.089832    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:11.938528 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:11.938495    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:12.702383 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:12.702353    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:13.591440 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:13.591410    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:14.342478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:14.342447    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:15.302446 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:15.302397    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:16.276011 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:16.275976    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:16.792417 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:16.792364    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:17.663883 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:17.663845    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:18.483391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:18.483353    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:19.276478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:19.276441    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:20.214977 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:20.214940    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:20.639715 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:20.639678    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:21.010434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:21.010396    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:21.252259 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:21.252225    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:21.360349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:21.360273    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:21.472389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:21.472353    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:22.145528 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.145477    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:22.623653 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.623617    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"]
Oct 30 20:29:22.624156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.624014    6389 cpu_manager.go:414] "containerMap: removing container" podUID="f1888692-4ce9-4e60-a913-0197d50e2084" containerName="revoke-keys"
Oct 30 20:29:22.624156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.624027    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1888692-4ce9-4e60-a913-0197d50e2084" containerName="revoke-keys"
Oct 30 20:29:22.624156 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.624127    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1888692-4ce9-4e60-a913-0197d50e2084" containerName="revoke-keys"
Oct 30 20:29:22.627527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.627505    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.637707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.637683    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"]
Oct 30 20:29:22.638278 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.638249    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-402199\"/\"openshift-service-ca.crt\""
Oct 30 20:29:22.638424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.638256    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-402199\"/\"kube-root-ca.crt\""
Oct 30 20:29:22.638424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.638316    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-402199\"/\"default-dockercfg-dn47g\""
Oct 30 20:29:22.638424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.638336    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-402199\"/\"simulator-single-main-402199-kserve-self-signed-certs\""
Oct 30 20:29:22.671035 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.670979    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:22.782826 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.782793    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.782826 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.782826    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5jc5\" (UniqueName: \"kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.783077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.782922    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.783077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.782991    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.783077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.783026    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.783077 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.783061    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.883687 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.883594    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.883687 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.883651    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5jc5\" (UniqueName: \"kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.883928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.883707    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.883928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.883735    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.883928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.883773    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.883928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.883826    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.884313 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.884287    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.884433 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.884327    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.884433 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.884391    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.886356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.886330    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.886501 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.886469    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.896481 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.896458    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5jc5\" (UniqueName: \"kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5\") pod \"simulator-single-main-402199-kserve-758476fb4f-dxt8q\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") " pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.941658 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.941624    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:22.976984 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:22.976955    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:23.226052 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:23.226027    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"]
Oct 30 20:29:23.231227 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:29:23.230912    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67af6079_5b2f_4625_94b4_9fc70cc3611f.slice/crio-7083b5f5030d45a009759630ef17204621341378d8101cb4cfb74ba98029b9af WatchSource:0}: Error finding container 7083b5f5030d45a009759630ef17204621341378d8101cb4cfb74ba98029b9af: Status 404 returned error can't find the container with id 7083b5f5030d45a009759630ef17204621341378d8101cb4cfb74ba98029b9af
Oct 30 20:29:23.846834 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:23.846757    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:23.961826 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:23.961791    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" event={"ID":"67af6079-5b2f-4625-94b4-9fc70cc3611f","Type":"ContainerStarted","Data":"235ca955d4df494d60fc1c4f2a38c56b39ff2f7da7a6face603fe2d5ec8f114c"}
Oct 30 20:29:23.961994 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:23.961835    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" event={"ID":"67af6079-5b2f-4625-94b4-9fc70cc3611f","Type":"ContainerStarted","Data":"7083b5f5030d45a009759630ef17204621341378d8101cb4cfb74ba98029b9af"}
Oct 30 20:29:23.985302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:23.985257    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" podStartSLOduration=1.985244893 podStartE2EDuration="1.985244893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:29:23.983880136 +0000 UTC m=+1484.183453063" watchObservedRunningTime="2026-10-30 20:29:23.985244893 +0000 UTC m=+1484.184817820"
Oct 30 20:29:24.880994 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:24.880947    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:24.942098 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:24.942064    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:25.677409 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:25.677376    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:25.791477 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:29:25.791441    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:29:26.517435 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:26.517399    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:27.419255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:27.419224    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:28.128833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:28.128799    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:28.913398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:28.913360    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:29.597740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:29.597708    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:30.367842 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:30.367805    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:31.076511 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:31.076476    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:31.838345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:31.838315    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:32.563051 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:32.563018    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:32.942476 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:32.942401    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:32.954781 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:32.954752    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:33.022223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:33.022195    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:29:33.376759 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:33.376720    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:34.099444 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:34.099413    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:34.951637 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:34.951606    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:35.576205 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:35.576159    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:35.679693 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:35.679650    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:36.384993 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:36.384957    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:37.216223 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:37.216192    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:37.955447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:37.955409    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:38.855938 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:38.855906    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:39.683363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:39.683329    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:40.110815 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:40.110785    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:29:40.113849 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:40.113827    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:29:40.423466 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:40.423376    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:40.684987 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:40.684902    6389 scope.go:122] "RemoveContainer" containerID="456f6d86f3c565036f8dc25ad491f9b60afdab8a249a0d5eee8bfc6e6d3afae0"
Oct 30 20:29:40.693449 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:40.693421    6389 scope.go:122] "RemoveContainer" containerID="53bc87a2e2e55397f96ac993e683e52ca62b25f9653c64d9fd3ffe1321a6c8cb"
Oct 30 20:29:41.270694 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:41.270655    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:41.963618 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:41.963581    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:42.670577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:42.670535    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:43.439750 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:43.439717    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:44.362982 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:44.362945    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:45.440100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:45.440065    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:46.256434 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:46.256405    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:46.962633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:46.962602    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:47.763935 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:47.763897    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:48.462549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:48.462512    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:49.264504 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:49.264465    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:49.969015 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:49.968982    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:50.777562 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:50.777523    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:51.492072 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:51.492035    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:52.369923 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:52.369892    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:53.071722 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:53.071685    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:53.962437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:53.962407    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:54.077780 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:54.077744    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:54.974088 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:54.974050    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:55.024895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:55.024865    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"]
Oct 30 20:29:55.027899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:55.027868    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-ndpd9"]
Oct 30 20:29:55.683404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:55.683347    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:55.791155 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:29:55.791100    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:29:55.912767 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:55.912735    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c00cfc0-b126-47c0-9c6f-af1674603009" path="/var/lib/kubelet/pods/7c00cfc0-b126-47c0-9c6f-af1674603009/volumes"
Oct 30 20:29:56.653207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:56.653175    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:57.468778 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:57.468740    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:58.364246 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:58.364205    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:58.996991 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:58.996959    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"]
Oct 30 20:29:59.091064 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:59.091029    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:59.190740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:59.190710    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:29:59.831932 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:29:59.831903    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:00.144457 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.144377    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"]
Oct 30 20:30:00.147962 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.147932    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.163208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.163177    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"]
Oct 30 20:30:00.244350 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.244316    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wbqtx\""
Oct 30 20:30:00.319028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.318985    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvd6g\" (UniqueName: \"kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g\") pod \"maas-api-key-cleanup-29889870-hp8w8\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.319242 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.319173    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889870-hp8w8\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.420737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.420638    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvd6g\" (UniqueName: \"kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g\") pod \"maas-api-key-cleanup-29889870-hp8w8\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.420908 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.420746    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889870-hp8w8\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.421433 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.421411    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889870-hp8w8\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.433393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.433361    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvd6g\" (UniqueName: \"kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g\") pod \"maas-api-key-cleanup-29889870-hp8w8\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.473330 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.473300    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:00.664867 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.664832    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:00.750487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:00.750457    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"]
Oct 30 20:30:00.752483 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:30:00.752458    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f333618_5b36_470f_8cc4_bfbf47a9d6bf.slice/crio-99d5cbccac18c87566b13ace14a3cb9a8798ed95e37979c991c417294f4f07a7 WatchSource:0}: Error finding container 99d5cbccac18c87566b13ace14a3cb9a8798ed95e37979c991c417294f4f07a7: Status 404 returned error can't find the container with id 99d5cbccac18c87566b13ace14a3cb9a8798ed95e37979c991c417294f4f07a7
Oct 30 20:30:01.177923 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:01.177873    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerStarted","Data":"42071049aa84e2953cb4a0c8767c134e6aa0a53084a1b9e809cb0bfaef871e30"}
Oct 30 20:30:01.178387 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:01.177940    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerStarted","Data":"99d5cbccac18c87566b13ace14a3cb9a8798ed95e37979c991c417294f4f07a7"}
Oct 30 20:30:01.197150 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:01.197085    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" podStartSLOduration=1.197071138 podStartE2EDuration="1.197071138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:30:01.195778579 +0000 UTC m=+1521.395351508" watchObservedRunningTime="2026-10-30 20:30:01.197071138 +0000 UTC m=+1521.396644065"
Oct 30 20:30:01.391679 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:01.391594    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:02.170202 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:02.170136    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:02.184483 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:02.184454    6389 generic.go:356] "Generic (PLEG): container finished" podID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerID="42071049aa84e2953cb4a0c8767c134e6aa0a53084a1b9e809cb0bfaef871e30" exitCode=60
Oct 30 20:30:02.184831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:02.184517    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerDied","Data":"42071049aa84e2953cb4a0c8767c134e6aa0a53084a1b9e809cb0bfaef871e30"}
Oct 30 20:30:02.184831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:02.184780    6389 scope.go:122] "RemoveContainer" containerID="42071049aa84e2953cb4a0c8767c134e6aa0a53084a1b9e809cb0bfaef871e30"
Oct 30 20:30:02.977341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:02.977302    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:03.191189 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:03.191151    6389 generic.go:356] "Generic (PLEG): container finished" podID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerID="f892b3c2c22fc361e68ee538a43f10b0333103957e3e31c857137ed2f763a637" exitCode=60
Oct 30 20:30:03.191666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:03.191234    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerDied","Data":"f892b3c2c22fc361e68ee538a43f10b0333103957e3e31c857137ed2f763a637"}
Oct 30 20:30:03.191666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:03.191310    6389 scope.go:122] "RemoveContainer" containerID="42071049aa84e2953cb4a0c8767c134e6aa0a53084a1b9e809cb0bfaef871e30"
Oct 30 20:30:03.191666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:03.191522    6389 scope.go:122] "RemoveContainer" containerID="f892b3c2c22fc361e68ee538a43f10b0333103957e3e31c857137ed2f763a637"
Oct 30 20:30:03.191821 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:30:03.191773    6389 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889870-hp8w8_odh-ai-gateway-infra(3f333618-5b36-470f-8cc4-bfbf47a9d6bf)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf"
Oct 30 20:30:03.761060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:03.761019    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:04.198633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:04.198559    6389 scope.go:122] "RemoveContainer" containerID="f892b3c2c22fc361e68ee538a43f10b0333103957e3e31c857137ed2f763a637"
Oct 30 20:30:04.199054 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:30:04.198809    6389 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889870-hp8w8_odh-ai-gateway-infra(3f333618-5b36-470f-8cc4-bfbf47a9d6bf)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf"
Oct 30 20:30:04.410987 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:04.410948    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:04.511617 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:04.511570    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:04.611180 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:04.611109    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:05.242479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:05.242442    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:05.406137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:05.406093    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:05.506443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:05.506319    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:05.975207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:05.975174    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:11.992441 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:11.992406    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:12.091241 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:12.091206    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:12.443435 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:12.443353    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:14.017437 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.017390    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" podUID="67af6079-5b2f-4625-94b4-9fc70cc3611f" containerName="main" containerID="cri-o://235ca955d4df494d60fc1c4f2a38c56b39ff2f7da7a6face603fe2d5ec8f114c" gracePeriod=45
Oct 30 20:30:14.021649 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.021621    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" podUID="67af6079-5b2f-4625-94b4-9fc70cc3611f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.115:8000/ready\": EOF"
Oct 30 20:30:14.255989 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.255958    6389 generic.go:356] "Generic (PLEG): container finished" podID="67af6079-5b2f-4625-94b4-9fc70cc3611f" containerID="235ca955d4df494d60fc1c4f2a38c56b39ff2f7da7a6face603fe2d5ec8f114c" exitCode=0
Oct 30 20:30:14.256153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.256007    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" event={"ID":"67af6079-5b2f-4625-94b4-9fc70cc3611f","Type":"ContainerDied","Data":"235ca955d4df494d60fc1c4f2a38c56b39ff2f7da7a6face603fe2d5ec8f114c"}
Oct 30 20:30:14.256153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.256044    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q" event={"ID":"67af6079-5b2f-4625-94b4-9fc70cc3611f","Type":"ContainerDied","Data":"7083b5f5030d45a009759630ef17204621341378d8101cb4cfb74ba98029b9af"}
Oct 30 20:30:14.256153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.256059    6389 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7083b5f5030d45a009759630ef17204621341378d8101cb4cfb74ba98029b9af"
Oct 30 20:30:14.266406 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.266384    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:30:14.346135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346050    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache\") pod \"67af6079-5b2f-4625-94b4-9fc70cc3611f\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") "
Oct 30 20:30:14.346135 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346109    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home\") pod \"67af6079-5b2f-4625-94b4-9fc70cc3611f\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") "
Oct 30 20:30:14.346356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346161    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm\") pod \"67af6079-5b2f-4625-94b4-9fc70cc3611f\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") "
Oct 30 20:30:14.346356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346213    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5\" (UniqueName: \"kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5\") pod \"67af6079-5b2f-4625-94b4-9fc70cc3611f\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") "
Oct 30 20:30:14.346356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346252    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir\") pod \"67af6079-5b2f-4625-94b4-9fc70cc3611f\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") "
Oct 30 20:30:14.346356 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346283    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs\" (UniqueName: \"kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs\") pod \"67af6079-5b2f-4625-94b4-9fc70cc3611f\" (UID: \"67af6079-5b2f-4625-94b4-9fc70cc3611f\") "
Oct 30 20:30:14.346577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346385    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache" pod "67af6079-5b2f-4625-94b4-9fc70cc3611f" (UID: "67af6079-5b2f-4625-94b4-9fc70cc3611f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:30:14.346577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346400    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home" pod "67af6079-5b2f-4625-94b4-9fc70cc3611f" (UID: "67af6079-5b2f-4625-94b4-9fc70cc3611f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:30:14.346577 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346506    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir" pod "67af6079-5b2f-4625-94b4-9fc70cc3611f" (UID: "67af6079-5b2f-4625-94b4-9fc70cc3611f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:30:14.346692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346626    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:14.346692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346653    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:14.346692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.346668    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:14.348399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.348373    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm" pod "67af6079-5b2f-4625-94b4-9fc70cc3611f" (UID: "67af6079-5b2f-4625-94b4-9fc70cc3611f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:30:14.348521 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.348504    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs" pod "67af6079-5b2f-4625-94b4-9fc70cc3611f" (UID: "67af6079-5b2f-4625-94b4-9fc70cc3611f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:30:14.348882 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.348868    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5" pod "67af6079-5b2f-4625-94b4-9fc70cc3611f" (UID: "67af6079-5b2f-4625-94b4-9fc70cc3611f"). InnerVolumeSpecName "kube-api-access-p5jc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:30:14.447207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.447177    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67af6079-5b2f-4625-94b4-9fc70cc3611f-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:14.447207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.447204    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5jc5\" (UniqueName: \"kubernetes.io/projected/67af6079-5b2f-4625-94b4-9fc70cc3611f-kube-api-access-p5jc5\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:14.447207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:14.447215    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67af6079-5b2f-4625-94b4-9fc70cc3611f-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:15.261487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:15.261457    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"
Oct 30 20:30:15.284230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:15.284200    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"]
Oct 30 20:30:15.288551 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:15.288529    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-402199/simulator-single-main-402199-kserve-758476fb4f-dxt8q"]
Oct 30 20:30:15.912748 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:15.912716    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67af6079-5b2f-4625-94b4-9fc70cc3611f" path="/var/lib/kubelet/pods/67af6079-5b2f-4625-94b4-9fc70cc3611f/volumes"
Oct 30 20:30:17.907601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:17.907569    6389 scope.go:122] "RemoveContainer" containerID="f892b3c2c22fc361e68ee538a43f10b0333103957e3e31c857137ed2f763a637"
Oct 30 20:30:18.278621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:18.278588    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerStarted","Data":"9eb376909be64530d161f527bcfa33286bba79559220566ebd553f3eb025fd15"}
Oct 30 20:30:19.285729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:19.285695    6389 generic.go:356] "Generic (PLEG): container finished" podID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerID="9eb376909be64530d161f527bcfa33286bba79559220566ebd553f3eb025fd15" exitCode=60
Oct 30 20:30:19.286187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:19.285767    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerDied","Data":"9eb376909be64530d161f527bcfa33286bba79559220566ebd553f3eb025fd15"}
Oct 30 20:30:19.286187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:19.285824    6389 scope.go:122] "RemoveContainer" containerID="f892b3c2c22fc361e68ee538a43f10b0333103957e3e31c857137ed2f763a637"
Oct 30 20:30:19.286187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:19.286141    6389 scope.go:122] "RemoveContainer" containerID="9eb376909be64530d161f527bcfa33286bba79559220566ebd553f3eb025fd15"
Oct 30 20:30:19.286388 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:30:19.286368    6389 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889870-hp8w8_odh-ai-gateway-infra(3f333618-5b36-470f-8cc4-bfbf47a9d6bf)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf"
Oct 30 20:30:19.312666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:19.312644    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"]
Oct 30 20:30:20.427158 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.427112    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:20.496399 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.496357    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca\") pod \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") "
Oct 30 20:30:20.496587 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.496413    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g\" (UniqueName: \"kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g\") pod \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\" (UID: \"3f333618-5b36-470f-8cc4-bfbf47a9d6bf\") "
Oct 30 20:30:20.496730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.496695    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca" pod "3f333618-5b36-470f-8cc4-bfbf47a9d6bf" (UID: "3f333618-5b36-470f-8cc4-bfbf47a9d6bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:30:20.498465 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.498440    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g" pod "3f333618-5b36-470f-8cc4-bfbf47a9d6bf" (UID: "3f333618-5b36-470f-8cc4-bfbf47a9d6bf"). InnerVolumeSpecName "kube-api-access-qvd6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:30:20.597443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.597359    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvd6g\" (UniqueName: \"kubernetes.io/projected/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-kube-api-access-qvd6g\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:20.597443 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:20.597388    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f333618-5b36-470f-8cc4-bfbf47a9d6bf-maas-api-service-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:30:21.298841 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:21.298805    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"
Oct 30 20:30:21.299019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:21.298841    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8" event={"ID":"3f333618-5b36-470f-8cc4-bfbf47a9d6bf","Type":"ContainerDied","Data":"99d5cbccac18c87566b13ace14a3cb9a8798ed95e37979c991c417294f4f07a7"}
Oct 30 20:30:21.299019 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:21.298904    6389 scope.go:122] "RemoveContainer" containerID="9eb376909be64530d161f527bcfa33286bba79559220566ebd553f3eb025fd15"
Oct 30 20:30:21.322042 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:21.322018    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"]
Oct 30 20:30:21.327970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:21.327946    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889870-hp8w8"]
Oct 30 20:30:21.912388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:21.912357    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" path="/var/lib/kubelet/pods/3f333618-5b36-470f-8cc4-bfbf47a9d6bf/volumes"
Oct 30 20:30:25.812080 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:30:25.812034    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:30:27.039143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:27.036365    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:30:27.039143 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:27.036445    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:30:27.329207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:27.326281    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:30:27.329207 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:27.326345    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:30:35.639702 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:35.639659    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:36.939310 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:36.939276    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:37.175534 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:37.175502    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:39.027177 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:39.027140    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"]
Oct 30 20:30:39.030498 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:39.030471    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4cw85"]
Oct 30 20:30:39.913733 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:39.913703    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1" path="/var/lib/kubelet/pods/dc5f02f4-b36f-4bdb-b882-55dc7ca90eb1/volumes"
Oct 30 20:30:40.720427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:40.720396    6389 scope.go:122] "RemoveContainer" containerID="ccbc14ad82c37e0bf27046313909cf61494ae26cd4a885da85e5ebc31ca067d4"
Oct 30 20:30:43.267338 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:43.267299    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:30:55.790555 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:30:55.790513    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:30:58.057902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:58.057866    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"]
Oct 30 20:30:58.065798 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:58.065773    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-g8q4n"]
Oct 30 20:30:59.912714 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:30:59.912680    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2f227e0-448c-4d42-90a5-53b7ecddd94f" path="/var/lib/kubelet/pods/b2f227e0-448c-4d42-90a5-53b7ecddd94f/volumes"
Oct 30 20:31:09.027866 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:09.027828    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"]
Oct 30 20:31:09.032713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:09.032687    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-x9qsm"]
Oct 30 20:31:09.912720 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:09.912683    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96d445fd-936e-492b-bc8f-d2c2c69e7ca8" path="/var/lib/kubelet/pods/96d445fd-936e-492b-bc8f-d2c2c69e7ca8/volumes"
Oct 30 20:31:17.214741 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:17.214708    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:31:25.790948 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:31:25.790911    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:31:31.040039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:31.040005    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"]
Oct 30 20:31:31.045435 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:31.045405    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-j92jn"]
Oct 30 20:31:31.913084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:31.913049    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3487c011-e5f3-44fe-875d-47c2cba8d10a" path="/var/lib/kubelet/pods/3487c011-e5f3-44fe-875d-47c2cba8d10a/volumes"
Oct 30 20:31:35.718236 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:35.718198    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:31:40.746862 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:40.746826    6389 scope.go:122] "RemoveContainer" containerID="ae23831a6b4b1bff7ca134849c113559e93c1c9f8fd71cdba5ccb6d1ab47c04a"
Oct 30 20:31:40.755601 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:40.755582    6389 scope.go:122] "RemoveContainer" containerID="7fef7f342b799918697d48a95f1e4cc73641940831d7c62b014790fd391bce9e"
Oct 30 20:31:42.025034 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:42.024999    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"]
Oct 30 20:31:42.029289 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:42.029254    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-xc477"]
Oct 30 20:31:43.913641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:43.913611    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8a711fe-57d4-4485-9a83-453f4b7c434f" path="/var/lib/kubelet/pods/b8a711fe-57d4-4485-9a83-453f4b7c434f/volumes"
Oct 30 20:31:55.793435 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:31:55.793348    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:31:56.389912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:56.389866    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:31:56.492260 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:31:56.492213    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:02.509490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:02.509456    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:02.611001 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:02.610969    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:04.026689 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:04.026655    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"]
Oct 30 20:32:04.031067 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:04.031031    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-glntk"]
Oct 30 20:32:05.912500 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:05.912458    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4db0090f-38cc-4d15-a579-bda480337bea" path="/var/lib/kubelet/pods/4db0090f-38cc-4d15-a579-bda480337bea/volumes"
Oct 30 20:32:22.023666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:22.023628    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"]
Oct 30 20:32:22.026513 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:22.026481    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-lpqv6"]
Oct 30 20:32:23.813870 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:23.813830    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:23.913199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:23.913170    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f27e5a87-43d3-4eed-af56-1b832dcedb31" path="/var/lib/kubelet/pods/f27e5a87-43d3-4eed-af56-1b832dcedb31/volumes"
Oct 30 20:32:24.283643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:24.283611    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:24.493543 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:24.493498    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:25.790667 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:32:25.790631    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:32:28.008512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:28.008467    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:28.209049 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:28.208987    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:29.191349 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:29.191317    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:29.297199 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:29.297167    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:29.409933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:29.409901    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:29.520427 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:29.520388    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:30.216722 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:30.216682    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:31.081651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:31.081617    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:31.798457 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:31.798409    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:32.645570 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:32.645536    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:33.370721 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:33.370690    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:34.208902 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:34.208859    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:34.962268 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:34.962227    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:35.768920 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:35.768891    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:35.870598 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:35.870559    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:36.529507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:36.529469    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:37.396342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:37.396307    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:38.213318 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:38.213280    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:39.025682 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:39.025646    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"]
Oct 30 20:32:39.029208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:39.029181    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-qc2qk"]
Oct 30 20:32:39.142420 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:39.142390    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:39.913859 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:39.913819    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65093664-23bd-476e-896e-2b04c8ebd620" path="/var/lib/kubelet/pods/65093664-23bd-476e-896e-2b04c8ebd620/volumes"
Oct 30 20:32:40.059928 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:40.059893    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:40.764426 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:40.764390    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:40.782643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:40.782614    6389 scope.go:122] "RemoveContainer" containerID="6e33c5cb9f683dfef3e0e549c8bdd4490718ea26af02ae5cddfca8ebb961ddb8"
Oct 30 20:32:40.791104 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:40.791082    6389 scope.go:122] "RemoveContainer" containerID="04e1739553a8b1d218208b9db6d0f13c86188e0536fc0caacd8e915779789e62"
Oct 30 20:32:40.800769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:40.800742    6389 scope.go:122] "RemoveContainer" containerID="1679998b52ad4ff2b2be61b459c352ee8b596e2088db6cc50d1cdb297a14d56f"
Oct 30 20:32:40.811039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:40.811016    6389 scope.go:122] "RemoveContainer" containerID="974b3063c2db5c8b5ba81b58a9ffecfed66447d52460cd9ff9685e5674b59bf4"
Oct 30 20:32:41.567335 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:41.567302    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:42.462690 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:42.462657    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:43.094097 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:43.094064    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:43.194947 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:43.194917    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:43.774377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:43.774347    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:43.975628 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:43.975595    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:44.077402 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.077321    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:44.186240 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.186209    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:44.825537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825494    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"]
Oct 30 20:32:44.825930 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825911    6389 cpu_manager.go:414] "containerMap: removing container" podUID="67af6079-5b2f-4625-94b4-9fc70cc3611f" containerName="main"
Oct 30 20:32:44.826021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825932    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67af6079-5b2f-4625-94b4-9fc70cc3611f" containerName="main"
Oct 30 20:32:44.826021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825954    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:32:44.826021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825962    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:32:44.826021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825974    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:32:44.826021 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.825982    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:32:44.826304 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.826079    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:32:44.826304 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.826095    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="67af6079-5b2f-4625-94b4-9fc70cc3611f" containerName="main"
Oct 30 20:32:44.826304 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.826110    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:32:44.831169 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.831143    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.841623 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.841599    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-3f7c44\"/\"openshift-service-ca.crt\""
Oct 30 20:32:44.841740 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.841707    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-3f7c44\"/\"kube-root-ca.crt\""
Oct 30 20:32:44.841797 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.841783    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-3f7c44\"/\"simulator-single-main-3f7c44-kserve-self-signed-certs\""
Oct 30 20:32:44.841863 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.841849    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-3f7c44\"/\"default-dockercfg-nldjm\""
Oct 30 20:32:44.842277 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.842256    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"]
Oct 30 20:32:44.865507 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.865465    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.865650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.865528    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.865650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.865592    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.865650 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.865641    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.865760 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.865704    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch7gq\" (UniqueName: \"kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.865835 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.865816    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.949633 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.949596    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:44.966475 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966446    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966485    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966512    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966535    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966557    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch7gq\" (UniqueName: \"kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966595    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966903 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966879    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.966942 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.966934    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.967040 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.967022    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.968891 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.968865    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.969208 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.969191    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:44.979573 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:44.979552    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch7gq\" (UniqueName: \"kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq\") pod \"simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") " pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:45.144322 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:45.144239    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:45.230815 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:45.230752    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:45.430167 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:45.430067    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"]
Oct 30 20:32:45.434004 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:32:45.433979    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod620259de_660d_4f60_ab65_13ee01fb0f6c.slice/crio-1e8116a8999977cb4b1239bbfd2c5bda43b9214eb02be2540b600fcca0507bff WatchSource:0}: Error finding container 1e8116a8999977cb4b1239bbfd2c5bda43b9214eb02be2540b600fcca0507bff: Status 404 returned error can't find the container with id 1e8116a8999977cb4b1239bbfd2c5bda43b9214eb02be2540b600fcca0507bff
Oct 30 20:32:46.040938 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:46.040902    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:46.124579 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:46.124539    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd" event={"ID":"620259de-660d-4f60-ab65-13ee01fb0f6c","Type":"ContainerStarted","Data":"cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953"}
Oct 30 20:32:46.124774 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:46.124592    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd" event={"ID":"620259de-660d-4f60-ab65-13ee01fb0f6c","Type":"ContainerStarted","Data":"1e8116a8999977cb4b1239bbfd2c5bda43b9214eb02be2540b600fcca0507bff"}
Oct 30 20:32:46.144564 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:46.144543    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:46.147856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:46.147812    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd" podStartSLOduration=2.147797708 podStartE2EDuration="2.147797708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:32:46.147008094 +0000 UTC m=+1686.346581023" watchObservedRunningTime="2026-10-30 20:32:46.147797708 +0000 UTC m=+1686.347370691"
Oct 30 20:32:46.976451 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:46.976420    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:47.883384 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:47.883335    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:48.660702 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:48.660673    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:49.654905 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:49.654870    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:50.465553 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:50.465509    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:51.300432 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:51.300398    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:52.097986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:52.097943    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:52.804697 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:52.804645    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:53.766789 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:53.766755    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:54.579856 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:54.579829    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:55.145360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:55.145327    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:55.157606 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:55.157581    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:55.181607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:55.181582    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:32:55.456830 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:55.456758    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:55.791608 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:32:55.791573    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:32:56.158164 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:56.158059    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:57.028327 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:57.028296    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"]
Oct 30 20:32:57.028518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:57.028493    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:57.031600 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:57.031576    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-qd46n"]
Oct 30 20:32:57.756527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:57.756492    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:57.912561 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:57.912529    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d3e19c9-e5a8-4ba6-859c-bdc35457747a" path="/var/lib/kubelet/pods/4d3e19c9-e5a8-4ba6-859c-bdc35457747a/volumes"
Oct 30 20:32:58.521113 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:58.521067    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:59.252973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:59.252938    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:32:59.958438 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:32:59.958409    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:00.773885 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:00.773852    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:01.550081 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:01.550046    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:02.452769 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:02.452736    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:03.363786 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:03.363752    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:04.148520 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:04.148484    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:04.855651 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:04.855616    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:05.573620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:05.573590    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:06.295709 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:06.295673    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:07.006044 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:07.006011    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:07.855393 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:07.855356    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:08.563106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:08.563071    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:09.365978 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:09.365942    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:10.052718 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:10.052676    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:10.862000 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:10.861960    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:11.659548 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:11.659515    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:12.349939 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:12.349905    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:13.175817 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:13.175785    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:13.696912 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:13.696874    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:13.942321 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:13.942288    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:14.857871 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:14.857842    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:15.638815 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:15.638774    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:16.569607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:16.569568    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:17.275100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:17.275067    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:18.155648 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:18.155614    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:18.656620 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:18.656583    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"]
Oct 30 20:33:18.731692 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:18.731654    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:18.853744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:18.853708    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:19.645366 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:19.645333    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:20.352254 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:20.352220    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:21.053544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:21.053465    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:21.866357 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:21.866324    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:22.660783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:22.660737    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:23.352527 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:23.352496    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:23.948985 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:23.948943    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:24.050621 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:24.050593    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:24.408663 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:24.408631    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:24.520668 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:24.520635    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:24.977832 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:24.977803    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:25.791218 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:33:25.791182    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:33:30.534106 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:30.534070    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:31.227915 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:31.227884    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:31.328957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:31.328927    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:31.634810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:31.634729    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:33.677003 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:33.676960    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd" podUID="620259de-660d-4f60-ab65-13ee01fb0f6c" containerName="main" containerID="cri-o://cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953" gracePeriod=45
Oct 30 20:33:33.920137 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:33.920095    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:33:34.010963 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.010933    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq\" (UniqueName: \"kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq\") pod \"620259de-660d-4f60-ab65-13ee01fb0f6c\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") "
Oct 30 20:33:34.011139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.010981    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm\") pod \"620259de-660d-4f60-ab65-13ee01fb0f6c\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") "
Oct 30 20:33:34.011139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011019    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs\" (UniqueName: \"kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs\") pod \"620259de-660d-4f60-ab65-13ee01fb0f6c\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") "
Oct 30 20:33:34.011139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011043    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache\") pod \"620259de-660d-4f60-ab65-13ee01fb0f6c\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") "
Oct 30 20:33:34.011139 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011099    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir\") pod \"620259de-660d-4f60-ab65-13ee01fb0f6c\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") "
Oct 30 20:33:34.011358 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011182    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home\") pod \"620259de-660d-4f60-ab65-13ee01fb0f6c\" (UID: \"620259de-660d-4f60-ab65-13ee01fb0f6c\") "
Oct 30 20:33:34.011430 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011362    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache" pod "620259de-660d-4f60-ab65-13ee01fb0f6c" (UID: "620259de-660d-4f60-ab65-13ee01fb0f6c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:33:34.011487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011444    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir" pod "620259de-660d-4f60-ab65-13ee01fb0f6c" (UID: "620259de-660d-4f60-ab65-13ee01fb0f6c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:33:34.011525 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011506    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home" pod "620259de-660d-4f60-ab65-13ee01fb0f6c" (UID: "620259de-660d-4f60-ab65-13ee01fb0f6c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:33:34.011938 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.011916    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:33:34.012148 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.012105    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:33:34.012271 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.012152    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:33:34.013410 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.013376    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm" pod "620259de-660d-4f60-ab65-13ee01fb0f6c" (UID: "620259de-660d-4f60-ab65-13ee01fb0f6c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:33:34.013410 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.013390    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs" pod "620259de-660d-4f60-ab65-13ee01fb0f6c" (UID: "620259de-660d-4f60-ab65-13ee01fb0f6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:33:34.013536 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.013466    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq" pod "620259de-660d-4f60-ab65-13ee01fb0f6c" (UID: "620259de-660d-4f60-ab65-13ee01fb0f6c"). InnerVolumeSpecName "kube-api-access-ch7gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:33:34.112970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.112940    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch7gq\" (UniqueName: \"kubernetes.io/projected/620259de-660d-4f60-ab65-13ee01fb0f6c-kube-api-access-ch7gq\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:33:34.112970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.112967    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/620259de-660d-4f60-ab65-13ee01fb0f6c-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:33:34.112970 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.112977    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/620259de-660d-4f60-ab65-13ee01fb0f6c-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:33:34.372661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.372581    6389 generic.go:356] "Generic (PLEG): container finished" podID="620259de-660d-4f60-ab65-13ee01fb0f6c" containerID="cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953" exitCode=0
Oct 30 20:33:34.372661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.372638    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"
Oct 30 20:33:34.372873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.372673    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd" event={"ID":"620259de-660d-4f60-ab65-13ee01fb0f6c","Type":"ContainerDied","Data":"cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953"}
Oct 30 20:33:34.372873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.372717    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd" event={"ID":"620259de-660d-4f60-ab65-13ee01fb0f6c","Type":"ContainerDied","Data":"1e8116a8999977cb4b1239bbfd2c5bda43b9214eb02be2540b600fcca0507bff"}
Oct 30 20:33:34.372873 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.372734    6389 scope.go:122] "RemoveContainer" containerID="cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953"
Oct 30 20:33:34.381853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.381835    6389 scope.go:122] "RemoveContainer" containerID="cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953"
Oct 30 20:33:34.382191 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:33:34.382165    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953\": container with ID starting with cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953 not found: ID does not exist" containerID="cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953"
Oct 30 20:33:34.382306 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.382199    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953"} err="failed to get container status \"cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953\": rpc error: code = NotFound desc = could not find container \"cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953\": container with ID starting with cf95332c2c46d1f1eba3d110e8d982241add2586aeacd2c93a81acd264eb9953 not found: ID does not exist"
Oct 30 20:33:34.397262 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.397242    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"]
Oct 30 20:33:34.399713 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:34.399683    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-3f7c44/simulator-single-main-3f7c44-kserve-7f75f4bb7c-pk7pd"]
Oct 30 20:33:35.855388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:35.855348    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:35.920853 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:35.920818    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="620259de-660d-4f60-ab65-13ee01fb0f6c" path="/var/lib/kubelet/pods/620259de-660d-4f60-ab65-13ee01fb0f6c/volumes"
Oct 30 20:33:36.356755 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:36.356716    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:36.457986 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:36.457948    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:37.517806 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.517765    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"]
Oct 30 20:33:37.518424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.518238    6389 cpu_manager.go:414] "containerMap: removing container" podUID="620259de-660d-4f60-ab65-13ee01fb0f6c" containerName="main"
Oct 30 20:33:37.518424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.518268    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="620259de-660d-4f60-ab65-13ee01fb0f6c" containerName="main"
Oct 30 20:33:37.518424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.518303    6389 cpu_manager.go:414] "containerMap: removing container" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:33:37.518424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.518310    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:33:37.518424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.518409    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="620259de-660d-4f60-ab65-13ee01fb0f6c" containerName="main"
Oct 30 20:33:37.518424 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.518428    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f333618-5b36-470f-8cc4-bfbf47a9d6bf" containerName="cleanup"
Oct 30 20:33:37.521891 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.521868    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.531883 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.531856    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-99bbafd5-kserve-self-signed-certs\""
Oct 30 20:33:37.532306 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.532287    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9h5g9\""
Oct 30 20:33:37.532467 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.532406    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 20:33:37.532572 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.532493    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 20:33:37.533178 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.533151    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"]
Oct 30 20:33:37.562495 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.562454    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:37.645919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.645884    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.645919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.645928    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.646230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.645957    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.646230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.646004    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.646230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.646026    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ccvw\" (UniqueName: \"kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.646230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.646049    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747348    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747389 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747394    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747507    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747545    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747567    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ccvw\" (UniqueName: \"kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747655 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747588    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747929    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.747957 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.747955    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.748071 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.748001    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.749744 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.749720    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.750039 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.750019    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.760479 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.760454    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ccvw\" (UniqueName: \"kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw\") pod \"e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") " pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:37.837470 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:37.837394    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:38.114170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:38.114070    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"]
Oct 30 20:33:38.117284 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:33:38.117256    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1da57c0d_2491_402c_be5b_35268f0eea9f.slice/crio-d9e333809a2b1e9049903ea808afa827f5c319dea5b8a4a75232db9acdb4feb5 WatchSource:0}: Error finding container d9e333809a2b1e9049903ea808afa827f5c319dea5b8a4a75232db9acdb4feb5: Status 404 returned error can't find the container with id d9e333809a2b1e9049903ea808afa827f5c319dea5b8a4a75232db9acdb4feb5
Oct 30 20:33:38.395643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:38.395563    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq" event={"ID":"1da57c0d-2491-402c-be5b-35268f0eea9f","Type":"ContainerStarted","Data":"ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700"}
Oct 30 20:33:38.395643 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:38.395604    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq" event={"ID":"1da57c0d-2491-402c-be5b-35268f0eea9f","Type":"ContainerStarted","Data":"d9e333809a2b1e9049903ea808afa827f5c319dea5b8a4a75232db9acdb4feb5"}
Oct 30 20:33:38.425992 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:38.425929    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq" podStartSLOduration=1.425911803 podStartE2EDuration="1.425911803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:33:38.424028151 +0000 UTC m=+1738.623601081" watchObservedRunningTime="2026-10-30 20:33:38.425911803 +0000 UTC m=+1738.625484732"
Oct 30 20:33:38.838272 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:38.838237    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:40.859140 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:40.858892    6389 scope.go:122] "RemoveContainer" containerID="37c5f68b972c212db01670a713ffd7273304ce295ff855db3c947e3032215c53"
Oct 30 20:33:40.911134 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:40.907300    6389 scope.go:122] "RemoveContainer" containerID="46641a540cf10fba4ee900fb724e96e3f129ad7e25079a3f09cb09afad13c423"
Oct 30 20:33:40.936490 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:40.931329    6389 scope.go:122] "RemoveContainer" containerID="363a0822224636f37d04fde1bf410d999d706cf2e3b5547bc08355d592212143"
Oct 30 20:33:42.035142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:42.034265    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:33:42.035142 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:42.034386    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:33:42.321341 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:42.321250    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:33:42.321615 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:42.321583    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:33:47.838552 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:47.838514    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:47.851288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:47.851260    6389 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:48.458235 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:48.458204    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:33:52.098550 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:52.098518    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:55.815484 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:33:55.815410    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:33:56.519281 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:56.519250    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:56.957100 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:56.957007    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:33:57.032247 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:57.032206    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:33:57.032603 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:57.032566    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:33:57.331153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:57.330243    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-2kmq5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:33:57.331153 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:33:57.330318    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-2kmq5" podUID="cf9639cd-63e1-4075-ac90-e4c5f7b25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:34:02.715667 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:02.715631    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:34:20.745145 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:20.745075    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"]
Oct 30 20:34:20.816737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:20.816699    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:34:22.820198 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:22.820161    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:34:25.793641 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:34:25.793597    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:34:31.435568 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:31.435530    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:34:31.537874 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:31.537837    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:34:35.765141 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:35.765084    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq" podUID="1da57c0d-2491-402c-be5b-35268f0eea9f" containerName="main" containerID="cri-o://ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700" gracePeriod=45
Oct 30 20:34:35.876609 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:35.876567    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:34:36.014419 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.014394    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:34:36.029316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029253    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs\" (UniqueName: \"kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs\") pod \"1da57c0d-2491-402c-be5b-35268f0eea9f\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") "
Oct 30 20:34:36.029316 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029287    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache\") pod \"1da57c0d-2491-402c-be5b-35268f0eea9f\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") "
Oct 30 20:34:36.029512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029317    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home\") pod \"1da57c0d-2491-402c-be5b-35268f0eea9f\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") "
Oct 30 20:34:36.029512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029347    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir\") pod \"1da57c0d-2491-402c-be5b-35268f0eea9f\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") "
Oct 30 20:34:36.029512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029382    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm\") pod \"1da57c0d-2491-402c-be5b-35268f0eea9f\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") "
Oct 30 20:34:36.029512 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029439    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw\" (UniqueName: \"kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw\") pod \"1da57c0d-2491-402c-be5b-35268f0eea9f\" (UID: \"1da57c0d-2491-402c-be5b-35268f0eea9f\") "
Oct 30 20:34:36.029743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029547    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache" pod "1da57c0d-2491-402c-be5b-35268f0eea9f" (UID: "1da57c0d-2491-402c-be5b-35268f0eea9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:34:36.029743 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029614    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home" pod "1da57c0d-2491-402c-be5b-35268f0eea9f" (UID: "1da57c0d-2491-402c-be5b-35268f0eea9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:34:36.029852 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029759    6389 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-model-cache\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:34:36.029852 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029779    6389 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-home\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:34:36.029965 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.029865    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir" pod "1da57c0d-2491-402c-be5b-35268f0eea9f" (UID: "1da57c0d-2491-402c-be5b-35268f0eea9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:34:36.031478 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.031455    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs" pod "1da57c0d-2491-402c-be5b-35268f0eea9f" (UID: "1da57c0d-2491-402c-be5b-35268f0eea9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:34:36.031607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.031555    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm" pod "1da57c0d-2491-402c-be5b-35268f0eea9f" (UID: "1da57c0d-2491-402c-be5b-35268f0eea9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:34:36.031607 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.031574    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw" pod "1da57c0d-2491-402c-be5b-35268f0eea9f" (UID: "1da57c0d-2491-402c-be5b-35268f0eea9f"). InnerVolumeSpecName "kube-api-access-8ccvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:34:36.130377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.130341    6389 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da57c0d-2491-402c-be5b-35268f0eea9f-tls-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:34:36.130377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.130379    6389 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-tmp-dir\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:34:36.130583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.130392    6389 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da57c0d-2491-402c-be5b-35268f0eea9f-dshm\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:34:36.130583 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.130407    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ccvw\" (UniqueName: \"kubernetes.io/projected/1da57c0d-2491-402c-be5b-35268f0eea9f-kube-api-access-8ccvw\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:34:36.744785 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.744748    6389 generic.go:356] "Generic (PLEG): container finished" podID="1da57c0d-2491-402c-be5b-35268f0eea9f" containerID="ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700" exitCode=0
Oct 30 20:34:36.744971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.744794    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"
Oct 30 20:34:36.744971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.744834    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq" event={"ID":"1da57c0d-2491-402c-be5b-35268f0eea9f","Type":"ContainerDied","Data":"ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700"}
Oct 30 20:34:36.744971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.744876    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq" event={"ID":"1da57c0d-2491-402c-be5b-35268f0eea9f","Type":"ContainerDied","Data":"d9e333809a2b1e9049903ea808afa827f5c319dea5b8a4a75232db9acdb4feb5"}
Oct 30 20:34:36.744971 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.744893    6389 scope.go:122] "RemoveContainer" containerID="ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700"
Oct 30 20:34:36.755363 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.755342    6389 scope.go:122] "RemoveContainer" containerID="ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700"
Oct 30 20:34:36.755646 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:34:36.755628    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700\": container with ID starting with ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700 not found: ID does not exist" containerID="ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700"
Oct 30 20:34:36.755707 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.755655    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700"} err="failed to get container status \"ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700\": rpc error: code = NotFound desc = could not find container \"ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700\": container with ID starting with ee798f8d5f401a503def6a81f7b8631aac0c6fa6dbe65cbededb728245580700 not found: ID does not exist"
Oct 30 20:34:36.773187 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.773166    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"]
Oct 30 20:34:36.777896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:36.777875    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-99bbafd5-kserve-7574467897-m9zkq"]
Oct 30 20:34:37.914776 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:37.914748    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1da57c0d-2491-402c-be5b-35268f0eea9f" path="/var/lib/kubelet/pods/1da57c0d-2491-402c-be5b-35268f0eea9f/volumes"
Oct 30 20:34:40.150333 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:40.150295    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:34:40.156729 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:40.156707    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnfz5_cdaa2197-530d-48e5-899c-5ccc58f4016a/ovn-acl-logging/0.log"
Oct 30 20:34:41.012060 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:41.012017    6389 scope.go:122] "RemoveContainer" containerID="1ee3e045dd5fa9213ae32aaf03c30170f8c04071b6b7677cefc8c3a8a9503b5b"
Oct 30 20:34:41.020831 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:41.020809    6389 scope.go:122] "RemoveContainer" containerID="cad60c6d33dfe9aa5be82d4badc1ef8a28bf4b12dc23fa9c43b70dc8f4a92c16"
Oct 30 20:34:55.790934 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:34:55.790845    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:34:59.905324 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.905290    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"]
Oct 30 20:34:59.905775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.905677    6389 cpu_manager.go:414] "containerMap: removing container" podUID="1da57c0d-2491-402c-be5b-35268f0eea9f" containerName="main"
Oct 30 20:34:59.905775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.905691    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1da57c0d-2491-402c-be5b-35268f0eea9f" containerName="main"
Oct 30 20:34:59.905775 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.905759    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="1da57c0d-2491-402c-be5b-35268f0eea9f" containerName="main"
Oct 30 20:34:59.910896 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.910874    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:34:59.923342 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.923316    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 20:34:59.923487 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.923392    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 20:34:59.923546 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.923490    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g52k9\""
Oct 30 20:34:59.923664 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:34:59.923636    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"]
Oct 30 20:35:00.015539 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.015497    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.015710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.015636    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.015710 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.015679    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.015833 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.015808    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxr2\" (UniqueName: \"kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.116641 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.116607    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.116810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.116657    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.116810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.116693    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.116810 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.116723    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxr2\" (UniqueName: \"kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.117388 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.117359    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.119099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.119073    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.119248 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.119159    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.129206 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.129179    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxr2\" (UniqueName: \"kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2\") pod \"maas-api-5c48877c75-z77kc\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.225028 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.225001    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:00.504170 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.504140    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"]
Oct 30 20:35:00.506330 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:35:00.506304    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a968fd_61d1_4522_994e_05eb8ea48914.slice/crio-3c0d055ee5ded770ad5d0ae41fefdac5732d60904c9adc2aff5cc1a7126ce05f WatchSource:0}: Error finding container 3c0d055ee5ded770ad5d0ae41fefdac5732d60904c9adc2aff5cc1a7126ce05f: Status 404 returned error can't find the container with id 3c0d055ee5ded770ad5d0ae41fefdac5732d60904c9adc2aff5cc1a7126ce05f
Oct 30 20:35:00.507553 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.507534    6389 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:35:00.896336 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:00.896250    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc" event={"ID":"42a968fd-61d1-4522-994e-05eb8ea48914","Type":"ContainerStarted","Data":"3c0d055ee5ded770ad5d0ae41fefdac5732d60904c9adc2aff5cc1a7126ce05f"}
Oct 30 20:35:01.904381 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:01.904349    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc" event={"ID":"42a968fd-61d1-4522-994e-05eb8ea48914","Type":"ContainerStarted","Data":"132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94"}
Oct 30 20:35:01.904782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:01.904484    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:01.925988 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:01.925944    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc" podStartSLOduration=2.435471404 podStartE2EDuration="2.925932679s" totalImagesPullingTime="490.461275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:35:01.92505986 +0000 UTC m=+1822.124632788" watchObservedRunningTime="2026-10-30 20:35:01.925932679 +0000 UTC m=+1822.125505607"
Oct 30 20:35:07.879047 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.879010    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"]
Oct 30 20:35:07.879684 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.879264    6389 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc" podUID="42a968fd-61d1-4522-994e-05eb8ea48914" containerName="maas-api" containerID="cri-o://132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94" gracePeriod=30
Oct 30 20:35:07.884186 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.884152    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:07.915974 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.915946    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-knbb4"]
Oct 30 20:35:07.919361 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.919339    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:07.930288 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.930249    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-knbb4"]
Oct 30 20:35:07.980197 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.980156    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-openshift-service-ca\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:07.980391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.980271    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhzps\" (UniqueName: \"kubernetes.io/projected/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-kube-api-access-bhzps\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:07.980391 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.980314    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-metrics-certs\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:07.980537 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:07.980413    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-maas-api-tls\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.081910 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.081875    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-maas-api-tls\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.082086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.081933    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-openshift-service-ca\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.082086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.081976    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhzps\" (UniqueName: \"kubernetes.io/projected/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-kube-api-access-bhzps\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.082086 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.081997    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-metrics-certs\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.082758 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.082731    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-openshift-service-ca\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.084659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.084611    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-maas-api-tls\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.085163 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.085140    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-metrics-certs\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.094782 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.094761    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhzps\" (UniqueName: \"kubernetes.io/projected/166b2ac6-239c-4e0a-a1aa-045d435dfd6f-kube-api-access-bhzps\") pod \"maas-api-595cc648c-knbb4\" (UID: \"166b2ac6-239c-4e0a-a1aa-045d435dfd6f\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.126518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.126488    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:08.184293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.183112    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca\") pod \"42a968fd-61d1-4522-994e-05eb8ea48914\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") "
Oct 30 20:35:08.184293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.183186    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls\") pod \"42a968fd-61d1-4522-994e-05eb8ea48914\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") "
Oct 30 20:35:08.184293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.183281    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs\") pod \"42a968fd-61d1-4522-994e-05eb8ea48914\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") "
Oct 30 20:35:08.184293 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.183350    6389 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2\" (UniqueName: \"kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2\") pod \"42a968fd-61d1-4522-994e-05eb8ea48914\" (UID: \"42a968fd-61d1-4522-994e-05eb8ea48914\") "
Oct 30 20:35:08.184636 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.184322    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca" pod "42a968fd-61d1-4522-994e-05eb8ea48914" (UID: "42a968fd-61d1-4522-994e-05eb8ea48914"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:35:08.185761 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.185727    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2" pod "42a968fd-61d1-4522-994e-05eb8ea48914" (UID: "42a968fd-61d1-4522-994e-05eb8ea48914"). InnerVolumeSpecName "kube-api-access-gtxr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:35:08.186022 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.185997    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs" pod "42a968fd-61d1-4522-994e-05eb8ea48914" (UID: "42a968fd-61d1-4522-994e-05eb8ea48914"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:35:08.186270 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.186255    6389 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls" pod "42a968fd-61d1-4522-994e-05eb8ea48914" (UID: "42a968fd-61d1-4522-994e-05eb8ea48914"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:35:08.273200 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.273165    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:08.284613 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.284589    6389 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-metrics-certs\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:35:08.284613 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.284611    6389 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtxr2\" (UniqueName: \"kubernetes.io/projected/42a968fd-61d1-4522-994e-05eb8ea48914-kube-api-access-gtxr2\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:35:08.284767 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.284625    6389 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a968fd-61d1-4522-994e-05eb8ea48914-openshift-service-ca\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:35:08.284767 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.284634    6389 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a968fd-61d1-4522-994e-05eb8ea48914-maas-api-tls\") on node \"ip-10-0-129-14.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:35:08.556157 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.555963    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-knbb4"]
Oct 30 20:35:08.559962 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:35:08.559934    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166b2ac6_239c_4e0a_a1aa_045d435dfd6f.slice/crio-6e48ab76efcbff33ead0665f659540c36ecc873abee3668b98fb28689ab159b4 WatchSource:0}: Error finding container 6e48ab76efcbff33ead0665f659540c36ecc873abee3668b98fb28689ab159b4: Status 404 returned error can't find the container with id 6e48ab76efcbff33ead0665f659540c36ecc873abee3668b98fb28689ab159b4
Oct 30 20:35:08.943859 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.943803    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4" event={"ID":"166b2ac6-239c-4e0a-a1aa-045d435dfd6f","Type":"ContainerStarted","Data":"6e48ab76efcbff33ead0665f659540c36ecc873abee3668b98fb28689ab159b4"}
Oct 30 20:35:08.945439 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.945410    6389 generic.go:356] "Generic (PLEG): container finished" podID="42a968fd-61d1-4522-994e-05eb8ea48914" containerID="132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94" exitCode=0
Oct 30 20:35:08.945595 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.945457    6389 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"
Oct 30 20:35:08.945595 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.945480    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc" event={"ID":"42a968fd-61d1-4522-994e-05eb8ea48914","Type":"ContainerDied","Data":"132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94"}
Oct 30 20:35:08.945595 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.945526    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-z77kc" event={"ID":"42a968fd-61d1-4522-994e-05eb8ea48914","Type":"ContainerDied","Data":"3c0d055ee5ded770ad5d0ae41fefdac5732d60904c9adc2aff5cc1a7126ce05f"}
Oct 30 20:35:08.945595 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.945558    6389 scope.go:122] "RemoveContainer" containerID="132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94"
Oct 30 20:35:08.965374 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.965303    6389 scope.go:122] "RemoveContainer" containerID="132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94"
Oct 30 20:35:08.966309 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:35:08.966278    6389 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94\": container with ID starting with 132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94 not found: ID does not exist" containerID="132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94"
Oct 30 20:35:08.966447 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.966318    6389 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94"} err="failed to get container status \"132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94\": rpc error: code = NotFound desc = could not find container \"132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94\": container with ID starting with 132370837dc77c8ded97b050d8521ab5c6a45b6265ca486521784ae258af0f94 not found: ID does not exist"
Oct 30 20:35:08.978351 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.978324    6389 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"]
Oct 30 20:35:08.982881 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:08.982856    6389 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-z77kc"]
Oct 30 20:35:09.914084 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:09.914050    6389 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42a968fd-61d1-4522-994e-05eb8ea48914" path="/var/lib/kubelet/pods/42a968fd-61d1-4522-994e-05eb8ea48914/volumes"
Oct 30 20:35:09.985144 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:09.984717    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4" event={"ID":"166b2ac6-239c-4e0a-a1aa-045d435dfd6f","Type":"ContainerStarted","Data":"51bf774ff0bdf8cf85038d9554b3109c561d3c6bef6d5d9f2e1562ef868c7656"}
Oct 30 20:35:09.985652 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:09.985634    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:10.008058 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:10.008009    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4" podStartSLOduration=2.5355986809999997 podStartE2EDuration="3.007998417s" totalImagesPullingTime="472.399736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:35:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:35:10.006704763 +0000 UTC m=+1830.206277704" watchObservedRunningTime="2026-10-30 20:35:10.007998417 +0000 UTC m=+1830.207571346"
Oct 30 20:35:16.999101 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:16.999072    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-knbb4"
Oct 30 20:35:25.791109 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:35:25.791068    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:35:32.951040 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:32.951004    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:35:35.932353 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:35.932322    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:35:41.120973 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:41.120940    6389 scope.go:122] "RemoveContainer" containerID="235ca955d4df494d60fc1c4f2a38c56b39ff2f7da7a6face603fe2d5ec8f114c"
Oct 30 20:35:55.817539 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:35:55.817503    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:35:57.032307 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:57.032255    6389 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-64d4cc998c-g2nbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:35:57.032814 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:35:57.032369    6389 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-64d4cc998c-g2nbq" podUID="8879a036-af1c-4d79-9897-633dd3a02edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:36:07.271214 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:07.271185    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-xv76h_e0b46f6b-b636-48f0-8bb6-822eee052780/manager/0.log"
Oct 30 20:36:07.493656 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:07.493625    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jgvzc_b9a99f11-b317-40da-8415-cced163bede5/manager/0.log"
Oct 30 20:36:08.397966 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:08.397932    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-p6lsk_35f63dc5-4733-491e-878a-2b30584bcd2a/manager/0.log"
Oct 30 20:36:09.113482 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:09.113450    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5_9f23eedf-3687-4fdf-a3e1-27d2455e0dbc/copy-manifests/0.log"
Oct 30 20:36:09.141784 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:09.141751    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5_9f23eedf-3687-4fdf-a3e1-27d2455e0dbc/manager/0.log"
Oct 30 20:36:10.358017 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:10.357975    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-knbb4_166b2ac6-239c-4e0a-a1aa-045d435dfd6f/maas-api/0.log"
Oct 30 20:36:12.268172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:12.268136    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-snrhr_d974ae8a-b8fd-4583-ab47-a3e6d408bd53/postgres/0.log"
Oct 30 20:36:14.510185 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:14.510155    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-x6hnt_50eaa387-2837-4f47-a710-139ad81cccf1/kuadrant-console-plugin/0.log"
Oct 30 20:36:15.186429 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:15.186397    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-lspfl_8d8ad07a-926f-47e2-a949-3dfcac0aae97/manager/0.log"
Oct 30 20:36:16.833964 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:16.833934    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-2kmq5_cf9639cd-63e1-4075-ac90-e4c5f7b25319/istio-proxy/0.log"
Oct 30 20:36:17.246160 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:17.246130    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-g2nbq_8879a036-af1c-4d79-9897-633dd3a02edf/istio-proxy/0.log"
Oct 30 20:36:18.015889 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:18.015864    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-ndkhp_3323ebee-b8a0-47a1-94d2-d04a8bed6ffc/istio-proxy/0.log"
Oct 30 20:36:25.790707 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:36:25.790673    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:36:32.482132 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:32.482083    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-xv76h_e0b46f6b-b636-48f0-8bb6-822eee052780/manager/0.log"
Oct 30 20:36:32.683168 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:32.683113    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jgvzc_b9a99f11-b317-40da-8415-cced163bede5/manager/0.log"
Oct 30 20:36:33.512464 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:33.512436    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-p6lsk_35f63dc5-4733-491e-878a-2b30584bcd2a/manager/0.log"
Oct 30 20:36:34.214783 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:34.214758    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5_9f23eedf-3687-4fdf-a3e1-27d2455e0dbc/copy-manifests/0.log"
Oct 30 20:36:34.242385 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:34.242360    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5_9f23eedf-3687-4fdf-a3e1-27d2455e0dbc/manager/0.log"
Oct 30 20:36:35.186043 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:35.186012    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-knbb4_166b2ac6-239c-4e0a-a1aa-045d435dfd6f/maas-api/0.log"
Oct 30 20:36:36.019895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:36.019862    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:36:37.027919 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:37.027890    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-snrhr_d974ae8a-b8fd-4583-ab47-a3e6d408bd53/postgres/0.log"
Oct 30 20:36:38.756224 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:38.756191    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-x6hnt_50eaa387-2837-4f47-a710-139ad81cccf1/kuadrant-console-plugin/0.log"
Oct 30 20:36:39.342670 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:39.342643    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-lspfl_8d8ad07a-926f-47e2-a949-3dfcac0aae97/manager/0.log"
Oct 30 20:36:40.651802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:40.651766    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-2kmq5_cf9639cd-63e1-4075-ac90-e4c5f7b25319/istio-proxy/0.log"
Oct 30 20:36:41.033705 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:41.033668    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-g2nbq_8879a036-af1c-4d79-9897-633dd3a02edf/istio-proxy/0.log"
Oct 30 20:36:41.779211 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:36:41.779160    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-ndkhp_3323ebee-b8a0-47a1-94d2-d04a8bed6ffc/istio-proxy/0.log"
Oct 30 20:36:55.791488 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:36:55.791450    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:37:25.792034 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:37:25.791995    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:37:26.089717 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:37:26.089620    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:35.945956 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:37:35.945923    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:36.079557 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:37:36.079522    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:36.179745 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:37:36.179718    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:55.792084 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:37:55.792010    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:38:12.007354 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:12.007322    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:12.122338 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:12.122305    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:15.281171 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:15.281141    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sgpdx_3ce05b8e-185a-48d6-b18b-5858b2959885/global-pull-secret-syncer/0.log"
Oct 30 20:38:15.532666 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:15.532566    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log"
Oct 30 20:38:15.655231 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:15.655199    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-14.us-east-2.compute.internal_bfb87cbb02f3462c0bb6e1279ac3fe5a/haproxy/0.log"
Oct 30 20:38:23.284062 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:23.284030    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-x6hnt_50eaa387-2837-4f47-a710-139ad81cccf1/kuadrant-console-plugin/0.log"
Oct 30 20:38:23.467302 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:23.467271    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-lspfl_8d8ad07a-926f-47e2-a949-3dfcac0aae97/manager/0.log"
Oct 30 20:38:24.609559 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.609531    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/alertmanager/0.log"
Oct 30 20:38:24.642500 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.642478    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/config-reloader/0.log"
Oct 30 20:38:24.670381 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.670356    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/kube-rbac-proxy-web/0.log"
Oct 30 20:38:24.697094 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.697074    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/kube-rbac-proxy/0.log"
Oct 30 20:38:24.723844 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.723826    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/kube-rbac-proxy-metric/0.log"
Oct 30 20:38:24.752404 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.752381    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/prom-label-proxy/0.log"
Oct 30 20:38:24.779172 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.779153    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fa1af69f-326f-491c-909a-7a63526052d6/init-config-reloader/0.log"
Oct 30 20:38:24.849423 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.849386    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-qcd5w_676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/kube-state-metrics/0.log"
Oct 30 20:38:24.873219 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.873144    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-qcd5w_676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/kube-rbac-proxy-main/0.log"
Oct 30 20:38:24.905377 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.905345    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-qcd5w_676bbe99-f475-40ee-9dfc-aa9cb8e8aa8e/kube-rbac-proxy-self/0.log"
Oct 30 20:38:24.968038 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:24.967995    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-4tl7l_a8d6d749-b397-476f-83ca-774cc1aa5a0d/monitoring-plugin/0.log"
Oct 30 20:38:25.008255 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.008233    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6vx6_26dcecb1-c6af-49e9-a840-b7197582b0cb/node-exporter/0.log"
Oct 30 20:38:25.033221 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.033202    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6vx6_26dcecb1-c6af-49e9-a840-b7197582b0cb/kube-rbac-proxy/0.log"
Oct 30 20:38:25.064398 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.064380    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6vx6_26dcecb1-c6af-49e9-a840-b7197582b0cb/init-textfile/0.log"
Oct 30 20:38:25.373726 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.373692    6389 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"]
Oct 30 20:38:25.374085 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.374073    6389 cpu_manager.go:414] "containerMap: removing container" podUID="42a968fd-61d1-4522-994e-05eb8ea48914" containerName="maas-api"
Oct 30 20:38:25.374212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.374087    6389 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42a968fd-61d1-4522-994e-05eb8ea48914" containerName="maas-api"
Oct 30 20:38:25.374212 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.374189    6389 memory_manager.go:365] "RemoveStaleState removing state" podUID="42a968fd-61d1-4522-994e-05eb8ea48914" containerName="maas-api"
Oct 30 20:38:25.377662 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.377640    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.387345 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.387324    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"]
Oct 30 20:38:25.398303 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.398273    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rlcdw_23c7a152-2247-4d92-9a1e-8f6a85b4954e/kube-rbac-proxy-main/0.log"
Oct 30 20:38:25.424866 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.424844    6389 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rnhfw\"/\"default-dockercfg-5cxtm\""
Oct 30 20:38:25.427018 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.426994    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rlcdw_23c7a152-2247-4d92-9a1e-8f6a85b4954e/kube-rbac-proxy-self/0.log"
Oct 30 20:38:25.436737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.436715    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-sys\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.436840 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.436779    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-podres\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.436890 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.436867    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-lib-modules\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.436933 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.436919    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-proc\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.436980 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.436963    6389 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r6f4\" (UniqueName: \"kubernetes.io/projected/a2820bb9-2eff-4c35-abaa-374b163e3e20-kube-api-access-5r6f4\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.462730 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.462701    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rlcdw_23c7a152-2247-4d92-9a1e-8f6a85b4954e/openshift-state-metrics/0.log"
Oct 30 20:38:25.478895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.478878    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rnhfw\"/\"openshift-service-ca.crt\""
Oct 30 20:38:25.478895 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.478901    6389 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rnhfw\"/\"kube-root-ca.crt\""
Oct 30 20:38:25.538382 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538356    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-lib-modules\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538352    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-lib-modules\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538433    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-proc\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538450    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r6f4\" (UniqueName: \"kubernetes.io/projected/a2820bb9-2eff-4c35-abaa-374b163e3e20-kube-api-access-5r6f4\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538518 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538480    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-sys\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538547    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-sys\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538561    6389 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-podres\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538609    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-podres\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.538716 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.538564    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2820bb9-2eff-4c35-abaa-374b163e3e20-proc\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.553911 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.553884    6389 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r6f4\" (UniqueName: \"kubernetes.io/projected/a2820bb9-2eff-4c35-abaa-374b163e3e20-kube-api-access-5r6f4\") pod \"perf-node-gather-daemonset-cfcch\" (UID: \"a2820bb9-2eff-4c35-abaa-374b163e3e20\") " pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.690779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.690693    6389 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:25.795541 ip-10-0-129-14 kubenswrapper[6389]: E1030 20:38:25.795493    6389 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-nnnfl" containerName="extractor"
Oct 30 20:38:25.965099 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:25.965076    6389 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"]
Oct 30 20:38:25.967758 ip-10-0-129-14 kubenswrapper[6389]: W1030 20:38:25.967730    6389 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2820bb9_2eff_4c35_abaa_374b163e3e20.slice/crio-6ee9e01023faa34028375b3e6c7c4ad866164e06e7a6d41fee4e4560948a27a4 WatchSource:0}: Error finding container 6ee9e01023faa34028375b3e6c7c4ad866164e06e7a6d41fee4e4560948a27a4: Status 404 returned error can't find the container with id 6ee9e01023faa34028375b3e6c7c4ad866164e06e7a6d41fee4e4560948a27a4
Oct 30 20:38:26.940950 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:26.940910    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch" event={"ID":"a2820bb9-2eff-4c35-abaa-374b163e3e20","Type":"ContainerStarted","Data":"faf96e2110e16cdd817e6961fc6e0617c87d6cf298817e5ad3bd60b8b8b6e5aa"}
Oct 30 20:38:26.941343 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:26.940967    6389 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch" event={"ID":"a2820bb9-2eff-4c35-abaa-374b163e3e20","Type":"ContainerStarted","Data":"6ee9e01023faa34028375b3e6c7c4ad866164e06e7a6d41fee4e4560948a27a4"}
Oct 30 20:38:26.941343 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:26.940990    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:26.960612 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:26.960568    6389 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch" podStartSLOduration=1.960556502 podStartE2EDuration="1.960556502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:38:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:26.960038242 +0000 UTC m=+2027.159611169" watchObservedRunningTime="2026-10-30 20:38:26.960556502 +0000 UTC m=+2027.160129430"
Oct 30 20:38:28.362802 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:28.362772    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-846957c67-krjg8_4c0f669d-17c7-4226-afe0-3593bc17a4da/console/0.log"
Oct 30 20:38:30.441230 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:30.441192    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t7h8t_dd239462-09eb-4071-83f6-171abe1cd346/dns/0.log"
Oct 30 20:38:30.469011 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:30.468971    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t7h8t_dd239462-09eb-4071-83f6-171abe1cd346/kube-rbac-proxy/0.log"
Oct 30 20:38:30.502457 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:30.502430    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzzv9_5efb380a-6530-4db0-88ec-2d59891344f1/dns-node-resolver/0.log"
Oct 30 20:38:31.273367 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:31.273333    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhts8_b5ddf8fa-c589-4ef9-8299-83ac15845b10/node-ca/0.log"
Oct 30 20:38:32.955779 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:32.955740    6389 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rnhfw/perf-node-gather-daemonset-cfcch"
Oct 30 20:38:33.701026 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:33.700995    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-2kmq5_cf9639cd-63e1-4075-ac90-e4c5f7b25319/istio-proxy/0.log"
Oct 30 20:38:33.772659 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:33.772635    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-g2nbq_8879a036-af1c-4d79-9897-633dd3a02edf/istio-proxy/0.log"
Oct 30 20:38:33.950549 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:33.950518    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-ndkhp_3323ebee-b8a0-47a1-94d2-d04a8bed6ffc/istio-proxy/0.log"
Oct 30 20:38:35.804588 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:35.804560    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ww6bn_81a971d1-9ef7-43cd-9f79-72ff95212079/serve-healthcheck-canary/0.log"
Oct 30 20:38:36.098267 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:36.098164    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:36.200929 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:36.200897    6389 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jh8v_eab05532-1d94-4218-bef2-a7f43bfd25e8/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:36.586634 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:36.586562    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nnnfl_ab48927b-5485-4685-84d1-6141d4de8ceb/kube-rbac-proxy/0.log"
Oct 30 20:38:36.614252 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:36.614201    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nnnfl_ab48927b-5485-4685-84d1-6141d4de8ceb/exporter/0.log"
Oct 30 20:38:36.642899 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:36.642876    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nnnfl_ab48927b-5485-4685-84d1-6141d4de8ceb/extractor/0.log"
Oct 30 20:38:37.214401 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:37.214375    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-2kmq5_cf9639cd-63e1-4075-ac90-e4c5f7b25319/istio-proxy/0.log"
Oct 30 20:38:37.258544 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:37.258517    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-g2nbq_8879a036-af1c-4d79-9897-633dd3a02edf/istio-proxy/0.log"
Oct 30 20:38:37.342284 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:37.342253    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-ndkhp_3323ebee-b8a0-47a1-94d2-d04a8bed6ffc/istio-proxy/0.log"
Oct 30 20:38:39.372322 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:39.372290    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-xv76h_e0b46f6b-b636-48f0-8bb6-822eee052780/manager/0.log"
Oct 30 20:38:39.703300 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:39.703190    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jgvzc_b9a99f11-b317-40da-8415-cced163bede5/manager/0.log"
Oct 30 20:38:40.091661 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:40.091612    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-p6lsk_35f63dc5-4733-491e-878a-2b30584bcd2a/manager/0.log"
Oct 30 20:38:40.307616 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:40.307586    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5_9f23eedf-3687-4fdf-a3e1-27d2455e0dbc/manager/0.log"
Oct 30 20:38:40.330851 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:40.330817    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cfd5cbdbb-lffr5_9f23eedf-3687-4fdf-a3e1-27d2455e0dbc/copy-manifests/0.log"
Oct 30 20:38:40.635261 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:40.635235    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-2kmq5_cf9639cd-63e1-4075-ac90-e4c5f7b25319/istio-proxy/0.log"
Oct 30 20:38:40.683360 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:40.683330    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-g2nbq_8879a036-af1c-4d79-9897-633dd3a02edf/istio-proxy/0.log"
Oct 30 20:38:40.785737 ip-10-0-129-14 kubenswrapper[6389]: I1030 20:38:40.785708    6389 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-64d4cc998c-ndkhp_3323ebee-b8a0-47a1-94d2-d04a8bed6ffc/istio-proxy/0.log"