Jul 02 19:03:39.977630 ip-10-0-135-120 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.448038    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458444    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458461    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458465    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458468    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458472    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458475    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458478    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:03:40.491756 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458481    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458485    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458488    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458491    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458494    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458497    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458499    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458502    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458505    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458508    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458511    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458514    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458517    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458520    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458523    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458525    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458528    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458531    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458534    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:03:40.492681 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458537    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458540    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458547    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458550    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458553    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458557    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458560    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458563    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458566    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458569    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458571    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458574    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458577    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458580    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458582    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458585    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458588    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458591    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458594    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:03:40.493985 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458597    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458602    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458606    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458609    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458612    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458623    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458626    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458629    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458632    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458635    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458638    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458641    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458644    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458647    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458650    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458652    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458655    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458658    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458662    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458664    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:03:40.494883 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458667    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458670    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458676    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458680    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458683    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458686    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458689    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458692    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458695    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458698    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458701    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458704    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458706    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458710    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458715    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458718    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458721    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458724    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458733    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:03:40.495552 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458736    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458740    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458743    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458746    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458749    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458752    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458755    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458757    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458760    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458763    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458766    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458768    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458771    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458774    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.458776    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461201    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461211    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461214    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461218    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461221    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:03:40.496639 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461224    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461227    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461231    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461234    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461237    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461240    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461243    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461246    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461249    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461252    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461254    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461258    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461261    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461264    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461267    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461272    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461275    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461277    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461280    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:03:40.497258 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461283    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461286    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461290    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461292    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461296    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461298    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461301    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461304    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461307    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461311    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461316    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461319    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461323    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461326    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461329    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461332    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461335    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461338    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461341    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461345    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:03:40.497924 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461348    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461351    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461354    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461357    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461360    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461366    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461370    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461373    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461394    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461397    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461400    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461406    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461409    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461412    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461415    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461417    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461421    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461423    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461426    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:03:40.498673 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461429    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461432    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461435    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461438    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461441    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461444    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461446    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461449    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461453    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461456    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461459    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461461    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461464    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461467    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461469    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461472    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461474    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461477    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461480    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461483    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:03:40.499278 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461486    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461489    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461492    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461495    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461500    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461504    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461507    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461511    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461514    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461516    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461519    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461522    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461524    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461527    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461530    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461533    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461931    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461936    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461939    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461942    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:03:40.499921 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461945    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461948    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461952    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461954    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461957    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461960    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461963    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461965    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461968    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461971    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461976    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461979    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461983    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461986    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461990    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461993    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.461997    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462001    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462004    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:03:40.598789 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462007    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:03:40.541527 ip-10-0-135-120 systemd[1]: Started Kubernetes Kubelet.
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462010    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462013    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462015    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462018    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462021    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462024    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462027    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462030    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462032    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462035    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462038    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462041    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462043    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462046    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462048    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462051    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462054    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462056    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462059    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:03:40.599717 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462062    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462065    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462068    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462071    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462074    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462077    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462080    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462083    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462086    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462090    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462092    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462095    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462098    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462101    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462104    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462106    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462109    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462112    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462114    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462117    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:03:40.600797 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462120    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462123    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462127    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462129    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462132    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462135    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462137    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462142    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462145    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462148    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462151    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462155    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462159    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462163    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462166    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462169    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462171    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462174    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462177    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:03:40.601531 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462180    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462183    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462187    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462189    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462192    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462195    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462197    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462200    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462203    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462205    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462208    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462211    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462213    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462216    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462219    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462221    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462224    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462303    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462310    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462317    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462321    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 19:03:40.602240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462325    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462329    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462333    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462337    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462341    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462346    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462350    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462353    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462356    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462360    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462363    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462366    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462369    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462374    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462378    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462381    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462384    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462387    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462391    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462394    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462398    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462401    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462404    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462407    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 19:03:40.602944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462410    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462413    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462435    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462439    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462443    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462447    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462450    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462453    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462456    2584 flags.go:64] FLAG: --enable-server="true"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462459    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462464    2584 flags.go:64] FLAG: --event-burst="100"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462468    2584 flags.go:64] FLAG: --event-qps="50"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462471    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462475    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462479    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462482    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462486    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462489    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462492    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462495    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462498    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462504    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462507    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462510    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462513    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 19:03:40.603765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462516    2584 flags.go:64] FLAG: --feature-gates=""
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462520    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462524    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462527    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462531    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462534    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462537    2584 flags.go:64] FLAG: --help="false"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462541    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462544    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462547    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462550    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462554    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462557    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462561    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462564    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462567    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462570    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462573    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462576    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462579    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462584    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462587    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462590    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462593    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 19:03:40.604609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462596    2584 flags.go:64] FLAG: --lock-file=""
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462599    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462603    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462606    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462613    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462617    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462620    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462623    2584 flags.go:64] FLAG: --logging-format="text"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462626    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462630    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462633    2584 flags.go:64] FLAG: --manifest-url=""
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462636    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462640    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462643    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462648    2584 flags.go:64] FLAG: --max-pods="110"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462651    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462654    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462657    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462660    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462664    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462667    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462670    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462678    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462681    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462684    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 19:03:40.605355 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462688    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462691    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462696    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462701    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462704    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462708    2584 flags.go:64] FLAG: --port="10250"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462711    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462714    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07ad6e6166ceaeb68"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462718    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462721    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462724    2584 flags.go:64] FLAG: --register-node="true"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462728    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462733    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462736    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462739    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462743    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462746    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462750    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462753    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462756    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462759    2584 flags.go:64] FLAG: --runonce="false"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462762    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462765    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462768    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462771    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 19:03:40.606192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462774    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462778    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462781    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462784    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462787    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462790    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462793    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462796    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462800    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462803    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462808    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462813    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462816    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462819    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462823    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462826    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462829    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462832    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462835    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462839    2584 flags.go:64] FLAG: --v="2"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462843    2584 flags.go:64] FLAG: --version="false"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462848    2584 flags.go:64] FLAG: --vmodule=""
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462852    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.462856    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462980    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:03:40.607034 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462984    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462988    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462991    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462993    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.462996    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463000    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463003    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463006    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463008    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463011    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463014    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463016    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463019    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463022    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463025    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463027    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463030    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463034    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463037    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463040    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:03:40.611463 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463042    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463045    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463048    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463050    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463053    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463056    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463060    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463063    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463066    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463073    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463075    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463078    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463081    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463084    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463086    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463089    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463092    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463095    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463097    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463100    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:03:40.612374 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463103    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463106    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463109    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463111    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463114    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463117    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463120    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463123    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463125    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463129    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463132    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463137    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463140    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463144    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463147    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463150    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463152    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463155    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:03:40.613276 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463159    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463161    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463164    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463167    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463170    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463173    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463176    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463178    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463181    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463184    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463186    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463189    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463192    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463196    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463200    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463203    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463206    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463209    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463212    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:03:40.614187 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463215    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463218    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463221    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463225    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463230    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463233    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463237    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463239    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463243    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463245    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463248    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463251    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463253    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463257    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463260    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463263    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463265    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463268    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463272    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:03:40.615075 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463275    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.463277    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.463879    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.477632    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.477648    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.477670    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.477677    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477750    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477756    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477761    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477765    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477769    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477772    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477776    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:03:40.616407 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477779    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477782    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477786    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477790    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477794    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477797    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477800    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477804    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477807    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477810    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477813    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477816    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477818    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477821    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477824    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477826    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477830    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477833    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477836    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:03:40.616987 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477840    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477843    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477846    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477850    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477853    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477856    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477859    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477862    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477864    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477867    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477870    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477873    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477876    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477878    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477881    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477885    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477889    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477892    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477895    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477897    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:03:40.617743 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477900    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477903    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477906    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477909    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477911    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477914    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477917    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477920    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477922    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477925    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477928    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477934    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477938    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477942    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477944    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477947    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477950    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477954    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477957    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477960    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:03:40.618340 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477963    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477966    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477968    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477971    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477974    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477977    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477980    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477982    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477985    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477988    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477991    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477994    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477996    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.477999    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478002    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478005    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478007    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478010    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478013    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:03:40.619046 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478015    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478018    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478021    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478023    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478028    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478032    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478035    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478038    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478041    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478043    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478052    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478055    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478058    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478061    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.478066    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:03:40.619805 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478213    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478219    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478222    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478225    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478227    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478230    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478233    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478236    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478238    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478241    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478244    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478247    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478250    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478253    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478255    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478258    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478261    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478263    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478268    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:03:40.702161 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478272    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478275    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478279    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478283    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478287    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478290    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478293    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478295    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478298    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478306    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478309    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478311    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478314    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478317    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478319    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478322    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478325    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478329    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478332    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:03:40.719938 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478335    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478338    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478341    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478344    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478347    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478349    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478352    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478355    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478358    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478360    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478363    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478366    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478369    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478372    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478375    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478379    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478382    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478386    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478388    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478391    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478394    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:03:40.721296 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478397    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478399    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478407    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478410    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478431    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478434    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478437    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478440    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478442    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478445    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478448    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478451    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478453    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478456    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478459    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478462    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478470    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478473    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478476    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478479    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:03:40.723847 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478482    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478485    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478488    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478490    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478493    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478496    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478500    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478503    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478507    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478511    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478514    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478518    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478521    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478524    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478526    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478534    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478537    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478540    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478543    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:03:40.724846 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:40.478546    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.478551    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.478676    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.480886    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.482312    2584 server.go:1019] "Starting client certificate rotation"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.482420    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.482485    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.486098    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.488361    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.505455    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.510087    2584 log.go:25] "Validated CRI v1 image API"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.511112    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 19:03:40.725692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.514396    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9a1ac33f-e401-4078-bbcd-7304325cfed9:/dev/nvme0n1p4 dacf6dea-22d8-4a9c-8523-0d05381148fa:/dev/nvme0n1p3]
Jul 02 19:03:40.726438 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.514430    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 19:03:40.726438 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.515878    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.519841    2584 manager.go:216] Machine: {Timestamp:2026-07-02 19:03:40.518466573 +0000 UTC m=+0.373398337 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099460 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec291f76d141511db3c8cb3429676f2f SystemUUID:ec291f76-d141-511d-b3c8-cb3429676f2f BootID:4671ce7e-75f1-4c40-a927-39e833341613 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:e5:48:49:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:e5:48:49:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:76:de:ce:38:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.519949    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.520033    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.524622    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.524657    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.525419    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.525432    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.525440    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.525455    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.526800    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529621    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529781    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529810    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529821    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529842    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529853    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.529868    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 19:03:40.727310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.530273    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.530291    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.532452    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.534477    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.534506    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535654    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535669    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535675    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535682    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535690    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535696    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535704    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535710    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535717    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535725    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535741    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.535747    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.536687    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.536700    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.540471    2584 server.go:1268] "Started kubelet"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.540902    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.540885    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.540956    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.542650    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.543207    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.544210    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nl7qd"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.547228    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.549679    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.550113    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.550115    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.550279    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.550296    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.550600    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nl7qd"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551215    2584 factory.go:55] Registering systemd factory
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551234    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551240    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.551215    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551249    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551536    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551679    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551691    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551743    2584 factory.go:153] Registering CRI-O factory
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551760    2584 factory.go:223] Registration of the crio container factory successfully
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551818    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551843    2584 factory.go:103] Registering Raw factory
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.551862    2584 manager.go:1196] Started watching for new ooms in manager
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.553105    2584 manager.go:349] Starting recovery of all containers
Jul 02 19:03:40.728850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.567626    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.569828    2584 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-120.ec2.internal\" not found" node="ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.576943    2584 manager.go:354] Recovery completed
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.581608    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.584808    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.584957    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.584969    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.585556    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.585568    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.585578    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.585588    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.586306    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.586333    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.586343    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.586995    2584 policy_none.go:49] "None policy: Start"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.587008    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.587019    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.588160    2584 policy_none.go:47] "Start"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642041    2584 manager.go:337] "Starting Device Plugin manager"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.642072    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642091    2584 server.go:87] "Starting device plugin registration server"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642370    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642382    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642499    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642577    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.642586    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.643185    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 19:03:40.730592 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.643240    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.743024    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.744373    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.744411    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.744426    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.744460    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.754594    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.754619    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-120.ec2.internal\": node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.769379    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.770612    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.770644    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.770664    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.770695    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.771607    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:40.830232 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.773272    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.872196    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.872209    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"]
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.872297    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.873319    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.873369    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.873383    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.874729    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.874852    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.874887    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.875586    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.875598    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.875606    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.875616    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.875620    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.875637    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.876827    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.876855    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.877550    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.877580    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.877596    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.917108    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-120.ec2.internal\" not found" node="ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.920836    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-120.ec2.internal\" not found" node="ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.953801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd10017bb1a5437629e392659cbc55f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal\" (UID: \"bd10017bb1a5437629e392659cbc55f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.953827    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f49a5dc7fed05757f7e38b0fbf5bfb5-config\") pod \"kube-apiserver-proxy-ip-10-0-135-120.ec2.internal\" (UID: \"6f49a5dc7fed05757f7e38b0fbf5bfb5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:40.953874    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd10017bb1a5437629e392659cbc55f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal\" (UID: \"bd10017bb1a5437629e392659cbc55f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:40.981299 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:40.972436    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.054971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd10017bb1a5437629e392659cbc55f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal\" (UID: \"bd10017bb1a5437629e392659cbc55f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.055009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd10017bb1a5437629e392659cbc55f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal\" (UID: \"bd10017bb1a5437629e392659cbc55f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.055029    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f49a5dc7fed05757f7e38b0fbf5bfb5-config\") pod \"kube-apiserver-proxy-ip-10-0-135-120.ec2.internal\" (UID: \"6f49a5dc7fed05757f7e38b0fbf5bfb5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.055076    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f49a5dc7fed05757f7e38b0fbf5bfb5-config\") pod \"kube-apiserver-proxy-ip-10-0-135-120.ec2.internal\" (UID: \"6f49a5dc7fed05757f7e38b0fbf5bfb5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.055083    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd10017bb1a5437629e392659cbc55f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal\" (UID: \"bd10017bb1a5437629e392659cbc55f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.055102    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd10017bb1a5437629e392659cbc55f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal\" (UID: \"bd10017bb1a5437629e392659cbc55f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.090002 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.073426    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.248274 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.174891    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.248274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.224832    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.248274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.224832    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.420042 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.276702    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.420042 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.378074    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.567611 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.479008    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.567611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.481265    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 19:03:41.567611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.481477    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 19:03:41.567611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.481477    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 19:03:41.567611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.553381    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 18:58:41 +0000 UTC" deadline="2027-12-31 16:34:41.579863615 +0000 UTC"
Jul 02 19:03:41.567611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.553414    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13125h31m0.026452975s"
Jul 02 19:03:41.675452 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.579660    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.682687 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.680224    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.780381 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:41.780338    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-120.ec2.internal\" not found"
Jul 02 19:03:41.789732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.789704    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:03:41.792081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.792061    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:03:41.851172 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.851137    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.865597 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.865574    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 19:03:41.866622 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.866607    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal"
Jul 02 19:03:41.878571 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.878549    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 19:03:41.915095 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:41.915064    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f49a5dc7fed05757f7e38b0fbf5bfb5.slice/crio-da4b0e1b428e07ce8e04045c86191afae501396b0fc1f0798e739f9bf3bcb5ec WatchSource:0}: Error finding container da4b0e1b428e07ce8e04045c86191afae501396b0fc1f0798e739f9bf3bcb5ec: Status 404 returned error can't find the container with id da4b0e1b428e07ce8e04045c86191afae501396b0fc1f0798e739f9bf3bcb5ec
Jul 02 19:03:41.915390 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:41.915355    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd10017bb1a5437629e392659cbc55f4.slice/crio-2e7783027930ceb984aeb0d75966e6c43d8234b6fdb0a8be19895e4bc0e7fa1d WatchSource:0}: Error finding container 2e7783027930ceb984aeb0d75966e6c43d8234b6fdb0a8be19895e4bc0e7fa1d: Status 404 returned error can't find the container with id 2e7783027930ceb984aeb0d75966e6c43d8234b6fdb0a8be19895e4bc0e7fa1d
Jul 02 19:03:41.919341 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:41.919324    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:03:42.457698 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.457661    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:03:42.531400 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.531087    2584 apiserver.go:52] "Watching apiserver"
Jul 02 19:03:42.538032 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.538003    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 19:03:42.538928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.538501    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-9sz77","openshift-multus/multus-jdkdc","openshift-multus/network-metrics-daemon-bj4fv","openshift-ovn-kubernetes/ovnkube-node-2p9d2","kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c","openshift-cluster-node-tuning-operator/tuned-9qn2x","openshift-image-registry/node-ca-sxpt2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal","openshift-multus/multus-additional-cni-plugins-m9qfc","openshift-network-diagnostics/network-check-target-2wgl5","openshift-network-operator/iptables-alerter-lpwqp","kube-system/global-pull-secret-syncer-hfqzn","kube-system/konnectivity-agent-d5vf8"]
Jul 02 19:03:42.540692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.540562    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.542778 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.542759    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:42.542880 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.542838    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:42.543569 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.543548    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.543669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.543601    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f4vbc\""
Jul 02 19:03:42.543669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.543640    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.544292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.544273    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:42.544376 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.544339    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:42.545603 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.545579    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.547782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.547656    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.548289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.548268    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 19:03:42.548558 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.548541    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.548846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.548829    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 19:03:42.549042 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.549023    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.549270 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.549254    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 19:03:42.549523 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.549507    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jfmgw\""
Jul 02 19:03:42.549723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.549705    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 19:03:42.550126 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.550106    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.550942 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.550924    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 19:03:42.551132 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.551116    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.553274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553254    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.553373 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553294    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 19:03:42.553466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553408    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.553590 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553551    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tg52x\""
Jul 02 19:03:42.553590 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553566    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.553714 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553686    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cj2xk\""
Jul 02 19:03:42.553803 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553748    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.553859 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553806    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 19:03:42.553911 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553875    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 19:03:42.553990 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.553970    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 19:03:42.556621 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.556597    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.556807 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.556719    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:42.557272 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.557247    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-slcjc\""
Jul 02 19:03:42.559361 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.559339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.562633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.561378    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.562633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.562257    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 19:03:42.562633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.562264    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.562633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.562313    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7szr2\""
Jul 02 19:03:42.562633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.562516    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.562903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.562808    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 19:03:42.564096 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.563703    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sj9gc\""
Jul 02 19:03:42.564096 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.563897    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.565469 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.565397    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.569497 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.569473    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 19:03:42.570116 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.569503    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 19:03:42.570116 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.569508    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.570315 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570297    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 19:03:42.570393 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570323    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.570393 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570374    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 19:03:42.570538 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570434    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 19:03:42.570538 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570450    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bxdk7\""
Jul 02 19:03:42.570801 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570778    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 19:03:42.570939 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.570779    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hzbsj\""
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589483    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5963213-fcb7-4e5d-a018-152633bf39e8-host\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589545    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-cni-bin\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5b75400-87e5-4124-a321-4852c2d15aec-konnectivity-ca\") pod \"konnectivity-agent-d5vf8\" (UID: \"a5b75400-87e5-4124-a321-4852c2d15aec\") " pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysconfig\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589682    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-tuned\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589723    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589753    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovnkube-config\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589819    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovn-node-metrics-cert\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589845    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysctl-d\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-sys\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-var-lib-kubelet\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.589990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70476c20-9439-43de-9b8b-a619dd8d5ddd-tmp\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590017    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.590104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590052    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-slash\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590088    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-kubernetes\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590118    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysctl-conf\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590147    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590173    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5963213-fcb7-4e5d-a018-152633bf39e8-serviceca\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590197    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d624d\" (UniqueName: \"kubernetes.io/projected/f5963213-fcb7-4e5d-a018-152633bf39e8-kube-api-access-d624d\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590224    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590249    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-ovn\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590275    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-log-socket\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-cni-netd\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-env-overrides\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590392    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-systemd\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-host\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590532    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kzd7\" (UniqueName: \"kubernetes.io/projected/cb1ea074-9270-4ef6-997c-b177aaee9982-kube-api-access-8kzd7\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-socket-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590602    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-kubelet\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591166 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-node-log\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590658    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m88g4\" (UniqueName: \"kubernetes.io/projected/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-kube-api-access-m88g4\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590684    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-kubelet-config\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590732    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-systemd-units\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590761    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-run-netns\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590817    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovnkube-script-lib\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590840    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r76n\" (UniqueName: \"kubernetes.io/projected/4d5ec31e-a746-4e9d-94c6-452d902740c8-kube-api-access-7r76n\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5b75400-87e5-4124-a321-4852c2d15aec-agent-certs\") pod \"konnectivity-agent-d5vf8\" (UID: \"a5b75400-87e5-4124-a321-4852c2d15aec\") " pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590887    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-device-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590913    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-var-lib-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590942    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-run\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.590991    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591022    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-sys-fs\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591092    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-dbus\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591131    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-systemd\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.591991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591155    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-etc-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.592864 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591174    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-modprobe-d\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.592864 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591207    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-lib-modules\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.592864 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9bn7\" (UniqueName: \"kubernetes.io/projected/70476c20-9439-43de-9b8b-a619dd8d5ddd-kube-api-access-v9bn7\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.592864 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.591267    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-registration-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.595892 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.595868    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nqgjz"
Jul 02 19:03:42.607027 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.607000    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nqgjz"
Jul 02 19:03:42.652495 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.652462    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 19:03:42.692095 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-systemd-units\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692095 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692100    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-run-netns\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692331 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692129    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692331 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r76n\" (UniqueName: \"kubernetes.io/projected/4d5ec31e-a746-4e9d-94c6-452d902740c8-kube-api-access-7r76n\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692331 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692195    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-systemd-units\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692331 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692251    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5b75400-87e5-4124-a321-4852c2d15aec-agent-certs\") pod \"konnectivity-agent-d5vf8\" (UID: \"a5b75400-87e5-4124-a321-4852c2d15aec\") " pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.692331 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-device-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-var-lib-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692408    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-cnibin\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692427    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-cni-multus\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-systemd\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-etc-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692492    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-modprobe-d\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-lib-modules\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692522    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-registration-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692521    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692537    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-daemon-config\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692570    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-cni-bin\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692580    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-run-netns\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692602    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysconfig\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.692625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692619    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-tuned\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692693    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-systemd\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692750    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692813    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-registration-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692915    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-modprobe-d\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692962    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-device-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693002    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693032    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c19b1252-7c1f-4ae2-86ad-47e5e4791628-cni-binary-copy\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693056    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693094    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsgqk\" (UniqueName: \"kubernetes.io/projected/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-kube-api-access-vsgqk\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693124    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovn-node-metrics-cert\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysctl-d\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-sys\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693231    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-var-lib-kubelet\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70476c20-9439-43de-9b8b-a619dd8d5ddd-tmp\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-conf-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.693411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693331    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6079c29a-a61e-4487-96c2-831e5f272a17-hosts-file\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693379    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5963213-fcb7-4e5d-a018-152633bf39e8-serviceca\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693406    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d624d\" (UniqueName: \"kubernetes.io/projected/f5963213-fcb7-4e5d-a018-152633bf39e8-kube-api-access-d624d\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-multus-certs\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-ovn\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693699    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-log-socket\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693740    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-cni-netd\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-env-overrides\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693809    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-systemd\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693837    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-hostroot\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693871    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-kubelet\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693901    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-kubelet-config\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693927    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-socket-dir-parent\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.692771    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-etc-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693954    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgzqg\" (UniqueName: \"kubernetes.io/projected/c19b1252-7c1f-4ae2-86ad-47e5e4791628-kube-api-access-rgzqg\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.693980    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.694288 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694001    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-run-ovn\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694014    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2smw\" (UniqueName: \"kubernetes.io/projected/a3820882-8598-456f-9604-0306e0956be5-kube-api-access-t2smw\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694048    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovnkube-script-lib\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694080    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-sys-fs\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-dbus\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694263    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-cni-bin\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694325    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysconfig\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694393    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-systemd\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694647    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-os-release\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694677    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-etc-kubernetes\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694707    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-run\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694734    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-netns\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-cni-bin\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694835    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9bn7\" (UniqueName: \"kubernetes.io/projected/70476c20-9439-43de-9b8b-a619dd8d5ddd-kube-api-access-v9bn7\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5963213-fcb7-4e5d-a018-152633bf39e8-host\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694893    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-k8s-cni-cncf-io\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.695161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694489    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-var-lib-openvswitch\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.694981    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5963213-fcb7-4e5d-a018-152633bf39e8-serviceca\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695032    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5963213-fcb7-4e5d-a018-152633bf39e8-host\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-os-release\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695081    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-sys\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695085    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-host-slash\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6079c29a-a61e-4487-96c2-831e5f272a17-tmp-dir\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695117    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-log-socket\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695155    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhqww\" (UniqueName: \"kubernetes.io/projected/6079c29a-a61e-4487-96c2-831e5f272a17-kube-api-access-hhqww\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695164    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-kubelet\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695185    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysctl-d\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695212    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-kubelet-config\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695239    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5b75400-87e5-4124-a321-4852c2d15aec-konnectivity-ca\") pod \"konnectivity-agent-d5vf8\" (UID: \"a5b75400-87e5-4124-a321-4852c2d15aec\") " pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695244    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-var-lib-kubelet\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695246    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-cni-netd\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695267    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-cnibin\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695303    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695338    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.696025 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.695360    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695370    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-iptables-alerter-script\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.695394    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695447    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-dbus\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.695463    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:43.195408854 +0000 UTC m=+3.050340622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695479    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695530    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-run\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.695571    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:43.195529221 +0000 UTC m=+3.050460977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695574    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695606    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovnkube-config\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695667    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-kubelet\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695669    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-lib-modules\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.695308    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-sys-fs\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696034    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-env-overrides\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.696937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696243    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5b75400-87e5-4124-a321-4852c2d15aec-konnectivity-ca\") pod \"konnectivity-agent-d5vf8\" (UID: \"a5b75400-87e5-4124-a321-4852c2d15aec\") " pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696243    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovnkube-script-lib\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696270    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-slash\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696329    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-host-slash\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-kubernetes\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696379    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysctl-conf\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696477    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-kubernetes\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696569    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-sysctl-conf\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-host\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kzd7\" (UniqueName: \"kubernetes.io/projected/cb1ea074-9270-4ef6-997c-b177aaee9982-kube-api-access-8kzd7\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696630    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovnkube-config\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696666    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-socket-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696689    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70476c20-9439-43de-9b8b-a619dd8d5ddd-host\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696697    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-system-cni-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696819    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-node-log\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.697833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696846    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m88g4\" (UniqueName: \"kubernetes.io/projected/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-kube-api-access-m88g4\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.698803 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696873    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-cni-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.698803 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.696898    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-system-cni-dir\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.698803 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.697008    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4d5ec31e-a746-4e9d-94c6-452d902740c8-node-log\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.698803 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.697040    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-socket-dir\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.700794 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.700765    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5b75400-87e5-4124-a321-4852c2d15aec-agent-certs\") pod \"konnectivity-agent-d5vf8\" (UID: \"a5b75400-87e5-4124-a321-4852c2d15aec\") " pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.700983 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.700965    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70476c20-9439-43de-9b8b-a619dd8d5ddd-etc-tuned\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.704313 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.704281    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d624d\" (UniqueName: \"kubernetes.io/projected/f5963213-fcb7-4e5d-a018-152633bf39e8-kube-api-access-d624d\") pod \"node-ca-sxpt2\" (UID: \"f5963213-fcb7-4e5d-a018-152633bf39e8\") " pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.706078 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.705406    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9bn7\" (UniqueName: \"kubernetes.io/projected/70476c20-9439-43de-9b8b-a619dd8d5ddd-kube-api-access-v9bn7\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.706078 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.705579    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:03:42.706078 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.705602    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:03:42.706078 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.705623    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:42.706078 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:42.705706    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:43.205676881 +0000 UTC m=+3.060608653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:42.708894 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.708833    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kzd7\" (UniqueName: \"kubernetes.io/projected/cb1ea074-9270-4ef6-997c-b177aaee9982-kube-api-access-8kzd7\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:42.711441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.711317    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m88g4\" (UniqueName: \"kubernetes.io/projected/7d8216ad-a1d3-46d7-9f79-0b36eb40443c-kube-api-access-m88g4\") pod \"aws-ebs-csi-driver-node-cnl2c\" (UID: \"7d8216ad-a1d3-46d7-9f79-0b36eb40443c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.712775 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.712717    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d5ec31e-a746-4e9d-94c6-452d902740c8-ovn-node-metrics-cert\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.712925 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.712896    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r76n\" (UniqueName: \"kubernetes.io/projected/4d5ec31e-a746-4e9d-94c6-452d902740c8-kube-api-access-7r76n\") pod \"ovnkube-node-2p9d2\" (UID: \"4d5ec31e-a746-4e9d-94c6-452d902740c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.713296 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.713209    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70476c20-9439-43de-9b8b-a619dd8d5ddd-tmp\") pod \"tuned-9qn2x\" (UID: \"70476c20-9439-43de-9b8b-a619dd8d5ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.778877 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.778816    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal" event={"ID":"6f49a5dc7fed05757f7e38b0fbf5bfb5","Type":"ContainerStarted","Data":"da4b0e1b428e07ce8e04045c86191afae501396b0fc1f0798e739f9bf3bcb5ec"}
Jul 02 19:03:42.780729 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.780667    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal" event={"ID":"bd10017bb1a5437629e392659cbc55f4","Type":"ContainerStarted","Data":"2e7783027930ceb984aeb0d75966e6c43d8234b6fdb0a8be19895e4bc0e7fa1d"}
Jul 02 19:03:42.798358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798165    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-system-cni-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-cni-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798245    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-system-cni-dir\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.798358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798277    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-cnibin\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-cni-multus\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798324    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-system-cni-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798335    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-daemon-config\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798378    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-system-cni-dir\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798402    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-cnibin\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798414    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c19b1252-7c1f-4ae2-86ad-47e5e4791628-cni-binary-copy\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798333    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-cni-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798427    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-cni-multus\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsgqk\" (UniqueName: \"kubernetes.io/projected/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-kube-api-access-vsgqk\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-conf-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6079c29a-a61e-4487-96c2-831e5f272a17-hosts-file\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798589    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-multus-certs\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-hostroot\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798650    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-socket-dir-parent\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798676    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzqg\" (UniqueName: \"kubernetes.io/projected/c19b1252-7c1f-4ae2-86ad-47e5e4791628-kube-api-access-rgzqg\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798705    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2smw\" (UniqueName: \"kubernetes.io/projected/a3820882-8598-456f-9604-0306e0956be5-kube-api-access-t2smw\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798764    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-os-release\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.798832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798806    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-etc-kubernetes\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798846    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-netns\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798872    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-cni-bin\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798901    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-k8s-cni-cncf-io\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798930    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-os-release\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798956    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-host-slash\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798983    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6079c29a-a61e-4487-96c2-831e5f272a17-tmp-dir\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.798998    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c19b1252-7c1f-4ae2-86ad-47e5e4791628-cni-binary-copy\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799007    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhqww\" (UniqueName: \"kubernetes.io/projected/6079c29a-a61e-4487-96c2-831e5f272a17-kube-api-access-hhqww\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799020    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799019    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-daemon-config\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799045    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-cnibin\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799124    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-netns\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799146    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-os-release\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799163    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-etc-kubernetes\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799195    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-cni-bin\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799211    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-cnibin\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799212    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-socket-dir-parent\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.799738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799231    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-k8s-cni-cncf-io\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799242    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6079c29a-a61e-4487-96c2-831e5f272a17-hosts-file\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799264    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-hostroot\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799288    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-host-slash\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799304    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-run-multus-certs\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799367    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-os-release\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799386    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3820882-8598-456f-9604-0306e0956be5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799456    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799456    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-multus-conf-dir\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-iptables-alerter-script\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799632    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-kubelet\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799687    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c19b1252-7c1f-4ae2-86ad-47e5e4791628-host-var-lib-kubelet\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.799772    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6079c29a-a61e-4487-96c2-831e5f272a17-tmp-dir\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.800001    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.800099    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-iptables-alerter-script\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.800735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.800268    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3820882-8598-456f-9604-0306e0956be5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.811921 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.811772    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2smw\" (UniqueName: \"kubernetes.io/projected/a3820882-8598-456f-9604-0306e0956be5-kube-api-access-t2smw\") pod \"multus-additional-cni-plugins-m9qfc\" (UID: \"a3820882-8598-456f-9604-0306e0956be5\") " pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.811921 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.811845    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsgqk\" (UniqueName: \"kubernetes.io/projected/f2e75684-2700-4cad-b7c8-43ec4b9bec6e-kube-api-access-vsgqk\") pod \"iptables-alerter-lpwqp\" (UID: \"f2e75684-2700-4cad-b7c8-43ec4b9bec6e\") " pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.812134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.812107    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgzqg\" (UniqueName: \"kubernetes.io/projected/c19b1252-7c1f-4ae2-86ad-47e5e4791628-kube-api-access-rgzqg\") pod \"multus-jdkdc\" (UID: \"c19b1252-7c1f-4ae2-86ad-47e5e4791628\") " pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.814053 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.814030    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhqww\" (UniqueName: \"kubernetes.io/projected/6079c29a-a61e-4487-96c2-831e5f272a17-kube-api-access-hhqww\") pod \"node-resolver-9sz77\" (UID: \"6079c29a-a61e-4487-96c2-831e5f272a17\") " pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.857407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.857296    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9qn2x"
Jul 02 19:03:42.868266 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.868239    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:03:42.870930 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.870902    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70476c20_9439_43de_9b8b_a619dd8d5ddd.slice/crio-8e90eacfd7495d0fc7ef4af80848823d613dcbe8257e66edef22e6d1b005a451 WatchSource:0}: Error finding container 8e90eacfd7495d0fc7ef4af80848823d613dcbe8257e66edef22e6d1b005a451: Status 404 returned error can't find the container with id 8e90eacfd7495d0fc7ef4af80848823d613dcbe8257e66edef22e6d1b005a451
Jul 02 19:03:42.877773 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.877731    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:42.879016 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.878988    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d5ec31e_a746_4e9d_94c6_452d902740c8.slice/crio-9eb141ea376f839951bf285af8d39b8d1fc26c0a83f75909eb9b106a8d6269cc WatchSource:0}: Error finding container 9eb141ea376f839951bf285af8d39b8d1fc26c0a83f75909eb9b106a8d6269cc: Status 404 returned error can't find the container with id 9eb141ea376f839951bf285af8d39b8d1fc26c0a83f75909eb9b106a8d6269cc
Jul 02 19:03:42.884921 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.884757    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c"
Jul 02 19:03:42.895037 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.895005    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sxpt2"
Jul 02 19:03:42.897813 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.897787    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b75400_87e5_4124_a321_4852c2d15aec.slice/crio-c82c17b49fe9692c190ec07bc72ab361d36e9fce3b4184a0d35c8823a7ab7f8e WatchSource:0}: Error finding container c82c17b49fe9692c190ec07bc72ab361d36e9fce3b4184a0d35c8823a7ab7f8e: Status 404 returned error can't find the container with id c82c17b49fe9692c190ec07bc72ab361d36e9fce3b4184a0d35c8823a7ab7f8e
Jul 02 19:03:42.903411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.903389    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jdkdc"
Jul 02 19:03:42.907504 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.906764    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:03:42.914162 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.914138    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m9qfc"
Jul 02 19:03:42.918345 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.918215    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19b1252_7c1f_4ae2_86ad_47e5e4791628.slice/crio-33581a63cb52bac7fd7d7908e628160982ea9172e5436a72fc7e71ddbfe3c2ba WatchSource:0}: Error finding container 33581a63cb52bac7fd7d7908e628160982ea9172e5436a72fc7e71ddbfe3c2ba: Status 404 returned error can't find the container with id 33581a63cb52bac7fd7d7908e628160982ea9172e5436a72fc7e71ddbfe3c2ba
Jul 02 19:03:42.922417 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.922382    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9sz77"
Jul 02 19:03:42.924428 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.924403    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3820882_8598_456f_9604_0306e0956be5.slice/crio-ecbe1ce9036da8955a2a90964e474b1da654736e12a0c805fa7499b12ffc1c3b WatchSource:0}: Error finding container ecbe1ce9036da8955a2a90964e474b1da654736e12a0c805fa7499b12ffc1c3b: Status 404 returned error can't find the container with id ecbe1ce9036da8955a2a90964e474b1da654736e12a0c805fa7499b12ffc1c3b
Jul 02 19:03:42.931371 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:42.931244    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lpwqp"
Jul 02 19:03:42.932083 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.932055    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6079c29a_a61e_4487_96c2_831e5f272a17.slice/crio-ca70b3d6f4e7053d291f401706569cc0e928c1bfd2fe021b2afa81d8fbeefcb9 WatchSource:0}: Error finding container ca70b3d6f4e7053d291f401706569cc0e928c1bfd2fe021b2afa81d8fbeefcb9: Status 404 returned error can't find the container with id ca70b3d6f4e7053d291f401706569cc0e928c1bfd2fe021b2afa81d8fbeefcb9
Jul 02 19:03:42.943804 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:03:42.943778    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e75684_2700_4cad_b7c8_43ec4b9bec6e.slice/crio-653e290bcb2a3b0da36864bbb239f7fdc93cb0f80c6e2ec3141fba8e08c1a052 WatchSource:0}: Error finding container 653e290bcb2a3b0da36864bbb239f7fdc93cb0f80c6e2ec3141fba8e08c1a052: Status 404 returned error can't find the container with id 653e290bcb2a3b0da36864bbb239f7fdc93cb0f80c6e2ec3141fba8e08c1a052
Jul 02 19:03:43.202994 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.202902    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:43.203191 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.202998    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:43.203191 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.203057    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:43.203191 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.203156    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:44.203136648 +0000 UTC m=+4.058068397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:43.203191 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.203165    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:43.203408 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.203221    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:44.203205177 +0000 UTC m=+4.058136929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:43.303787 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.303750    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:43.303998 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.303930    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:03:43.303998 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.303951    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:03:43.303998 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.303961    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:43.304159 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.304025    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:44.304005234 +0000 UTC m=+4.158937009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:43.610572 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.610431    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 18:58:42 +0000 UTC" deadline="2027-12-11 15:42:26.201624304 +0000 UTC"
Jul 02 19:03:43.610572 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.610474    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12644h38m42.591154082s"
Jul 02 19:03:43.771452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.771418    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:43.771452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.771451    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:43.771696 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.771556    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:43.771696 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.771644    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:43.771696 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.771633    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:43.771903 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:43.771748    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:43.788878 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.788762    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jdkdc" event={"ID":"c19b1252-7c1f-4ae2-86ad-47e5e4791628","Type":"ContainerStarted","Data":"33581a63cb52bac7fd7d7908e628160982ea9172e5436a72fc7e71ddbfe3c2ba"}
Jul 02 19:03:43.790543 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.790468    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sxpt2" event={"ID":"f5963213-fcb7-4e5d-a018-152633bf39e8","Type":"ContainerStarted","Data":"acaa72a434b4f736dcc43fb5a1130c567ab9d71b57ee97547197d31b875df198"}
Jul 02 19:03:43.792291 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.792223    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c" event={"ID":"7d8216ad-a1d3-46d7-9f79-0b36eb40443c","Type":"ContainerStarted","Data":"22580a3e05eccb95e9eeb4146840ade4842880c42e3d4e7331bf6886f85aaf2c"}
Jul 02 19:03:43.794176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.794150    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d5vf8" event={"ID":"a5b75400-87e5-4124-a321-4852c2d15aec","Type":"ContainerStarted","Data":"c82c17b49fe9692c190ec07bc72ab361d36e9fce3b4184a0d35c8823a7ab7f8e"}
Jul 02 19:03:43.795721 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.795693    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lpwqp" event={"ID":"f2e75684-2700-4cad-b7c8-43ec4b9bec6e","Type":"ContainerStarted","Data":"653e290bcb2a3b0da36864bbb239f7fdc93cb0f80c6e2ec3141fba8e08c1a052"}
Jul 02 19:03:43.797249 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.797219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"9eb141ea376f839951bf285af8d39b8d1fc26c0a83f75909eb9b106a8d6269cc"}
Jul 02 19:03:43.798792 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.798726    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9qn2x" event={"ID":"70476c20-9439-43de-9b8b-a619dd8d5ddd","Type":"ContainerStarted","Data":"8e90eacfd7495d0fc7ef4af80848823d613dcbe8257e66edef22e6d1b005a451"}
Jul 02 19:03:43.800028 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.799995    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9sz77" event={"ID":"6079c29a-a61e-4487-96c2-831e5f272a17","Type":"ContainerStarted","Data":"ca70b3d6f4e7053d291f401706569cc0e928c1bfd2fe021b2afa81d8fbeefcb9"}
Jul 02 19:03:43.802178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:43.802152    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerStarted","Data":"ecbe1ce9036da8955a2a90964e474b1da654736e12a0c805fa7499b12ffc1c3b"}
Jul 02 19:03:44.212081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:44.211988    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:44.212081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:44.212060    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:44.212318 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.212223    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:44.212318 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.212288    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:46.212268166 +0000 UTC m=+6.067199921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:44.212462 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.212352    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:44.212462 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.212404    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:46.212374882 +0000 UTC m=+6.067306638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:44.312564 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:44.312523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:44.312788 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.312771    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:03:44.312865 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.312802    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:03:44.312865 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.312816    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:44.312972 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:44.312878    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:46.312859052 +0000 UTC m=+6.167790820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:44.611034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:44.610878    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 18:58:42 +0000 UTC" deadline="2028-04-03 00:07:07.689247936 +0000 UTC"
Jul 02 19:03:44.611034 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:44.610946    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15365h3m23.07830648s"
Jul 02 19:03:45.772648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:45.771747    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:45.772648 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:45.771887    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:45.772648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:45.771963    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:45.772648 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:45.772040    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:45.772648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:45.772102    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:45.772648 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:45.772185    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:46.229616 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:46.229525    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:46.229616 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:46.229599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:46.229846 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.229792    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:46.229922 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.229910    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:50.229841099 +0000 UTC m=+10.084772852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:46.230376 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.230336    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:46.230494 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.230398    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:50.230381544 +0000 UTC m=+10.085313301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:46.331293 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:46.330658    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:46.331293 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.330817    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:03:46.331293 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.330836    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:03:46.331293 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.330850    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:46.331293 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:46.330900    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:50.330880727 +0000 UTC m=+10.185812491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:47.771189 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:47.771156    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:47.771639 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:47.771197    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:47.771639 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:47.771271    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:47.771639 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:47.771367    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:47.771639 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:47.771415    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:47.771639 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:47.771490    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:49.772130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:49.771591    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:49.772130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:49.771637    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:49.772130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:49.771592    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:49.772130 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:49.771746    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:49.772130 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:49.771807    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:49.772130 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:49.771879    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:50.262320 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:50.262291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:50.262453 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:50.262364    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:50.262453 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.262447    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:50.262568 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.262471    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:50.262568 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.262509    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:58.262490575 +0000 UTC m=+18.117422328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:50.262568 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.262528    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:58.262518725 +0000 UTC m=+18.117450473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:50.363038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:50.363000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:50.363223 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.363192    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:03:50.363223 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.363225    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:03:50.363409 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.363238    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:50.363409 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:50.363302    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:58.363282613 +0000 UTC m=+18.218214377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:51.772379 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:51.771313    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:51.772379 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:51.771687    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:51.772379 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:51.772095    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:51.772379 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:51.772183    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:51.772379 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:51.772244    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:51.772379 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:51.772320    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:53.771680 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:53.771617    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:53.772015 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:53.771738    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:53.772015 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:53.771770    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:53.772015 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:53.771868    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:53.772015 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:53.771904    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:53.772015 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:53.771989    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:53.828886 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:53.828860    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal" event={"ID":"6f49a5dc7fed05757f7e38b0fbf5bfb5","Type":"ContainerStarted","Data":"37a65bf7981cbeb4eb8bb4a8ade364f13b11b2b88a1f2c95113b687f4a237ed8"}
Jul 02 19:03:53.844490 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:53.844447    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-120.ec2.internal" podStartSLOduration=11.844434731 podStartE2EDuration="11.844434731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:53.844210346 +0000 UTC m=+13.699142116" watchObservedRunningTime="2026-07-02 19:03:53.844434731 +0000 UTC m=+13.699366500"
Jul 02 19:03:54.832063 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.831854    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9qn2x" event={"ID":"70476c20-9439-43de-9b8b-a619dd8d5ddd","Type":"ContainerStarted","Data":"db68150c7774dda2d78086b6988abe8f69811064271aef4e25346624903f9c71"}
Jul 02 19:03:54.834173 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.834141    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal" event={"ID":"bd10017bb1a5437629e392659cbc55f4","Type":"ContainerStarted","Data":"dcc8d70c92310bcd177461f35d6786c8bc7a61e5b3281c3ba6bc23c5911b7dbb"}
Jul 02 19:03:54.836021 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.835998    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9sz77" event={"ID":"6079c29a-a61e-4487-96c2-831e5f272a17","Type":"ContainerStarted","Data":"49c4c2caacf37b73336312bee03dc64d04faf5771f128b062fae6d3412852296"}
Jul 02 19:03:54.837889 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.837863    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerStarted","Data":"e80e9a37f657b8e45e9517f78abd7777058d920f325f887bc33e29235a568846"}
Jul 02 19:03:54.839714 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.839689    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sxpt2" event={"ID":"f5963213-fcb7-4e5d-a018-152633bf39e8","Type":"ContainerStarted","Data":"81057345b42f1388871b9c384a8f2853de38874a646a33e3368ed40db487cbf8"}
Jul 02 19:03:54.841473 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.841449    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c" event={"ID":"7d8216ad-a1d3-46d7-9f79-0b36eb40443c","Type":"ContainerStarted","Data":"756f1282392a59b2f01075d02c54bacd9e4a7fcc1e1ce2e6b05af624f68808f0"}
Jul 02 19:03:54.843495 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.843454    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d5vf8" event={"ID":"a5b75400-87e5-4124-a321-4852c2d15aec","Type":"ContainerStarted","Data":"235b962aa7f5296d2efa71572abd0844d3de8b6be0042083e25cf582aeedc8fe"}
Jul 02 19:03:54.848072 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.848027    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9qn2x" podStartSLOduration=2.955559209 podStartE2EDuration="13.848010976s" totalImagesPullingTime="10.892451767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:54.847520332 +0000 UTC m=+14.702452142" watchObservedRunningTime="2026-07-02 19:03:54.848010976 +0000 UTC m=+14.702942747"
Jul 02 19:03:54.863827 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.863778    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9sz77" podStartSLOduration=3.124362492 podStartE2EDuration="13.863763993s" totalImagesPullingTime="10.739401501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:54.863442344 +0000 UTC m=+14.718374113" watchObservedRunningTime="2026-07-02 19:03:54.863763993 +0000 UTC m=+14.718695763"
Jul 02 19:03:54.889404 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.889335    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sxpt2" podStartSLOduration=3.146302595 podStartE2EDuration="13.889314161s" totalImagesPullingTime="10.743011566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:54.888854196 +0000 UTC m=+14.743785968" watchObservedRunningTime="2026-07-02 19:03:54.889314161 +0000 UTC m=+14.744245936"
Jul 02 19:03:54.926143 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:54.926084    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d5vf8" podStartSLOduration=3.200712155 podStartE2EDuration="13.926068438s" totalImagesPullingTime="10.725356283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:54.925812435 +0000 UTC m=+14.780744260" watchObservedRunningTime="2026-07-02 19:03:54.926068438 +0000 UTC m=+14.781000209"
Jul 02 19:03:55.771186 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:55.771151    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:55.771385 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:55.771265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:55.771385 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:55.771278    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:55.771385 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:55.771287    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:55.771518 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:55.771448    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:55.771575 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:55.771547    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:55.846747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:55.846708    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lpwqp" event={"ID":"f2e75684-2700-4cad-b7c8-43ec4b9bec6e","Type":"ContainerStarted","Data":"e5a197b4069b1aa806bcf63be3d99118a8eda91448f57efdc184f8bba09916a4"}
Jul 02 19:03:55.862867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:55.862812    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lpwqp" podStartSLOduration=4.136655218 podStartE2EDuration="14.862794557s" totalImagesPullingTime="10.726139339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:55.86255362 +0000 UTC m=+15.717485390" watchObservedRunningTime="2026-07-02 19:03:55.862794557 +0000 UTC m=+15.717726326"
Jul 02 19:03:56.335240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:56.335203    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:56.335847 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:56.335831    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:56.848356 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:56.848308    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:56.848897 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:56.848825    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d5vf8"
Jul 02 19:03:57.770894 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:57.770847    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:57.771124 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:57.770955    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:57.771124 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:57.770967    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:57.771124 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:57.770981    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:57.771124 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:57.771070    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:03:57.771314 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:57.771179    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:58.329167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:58.329130    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:58.329747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:58.329194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:58.329747 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.329286    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:58.329747 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.329372    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:14.329353291 +0000 UTC m=+34.184285053 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:03:58.329747 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.329371    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:58.329747 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.329403    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:14.329396041 +0000 UTC m=+34.184327790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:03:58.430466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:58.430434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:58.430630 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.430597    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:03:58.430630 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.430618    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:03:58.430630 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.430628    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:58.430734 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:58.430681    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:14.430666338 +0000 UTC m=+34.285598085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:03:58.854585 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:58.854550    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3820882-8598-456f-9604-0306e0956be5" containerID="e80e9a37f657b8e45e9517f78abd7777058d920f325f887bc33e29235a568846" exitCode=0
Jul 02 19:03:58.854745 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:58.854634    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerDied","Data":"e80e9a37f657b8e45e9517f78abd7777058d920f325f887bc33e29235a568846"}
Jul 02 19:03:59.771466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:59.771432    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:03:59.771912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:59.771471    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:03:59.771912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:03:59.771470    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:03:59.771912 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:59.771560    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:03:59.771912 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:59.771670    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:03:59.771912 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:03:59.771778    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:01.771684 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:01.771644    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:01.771684 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:01.771666    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:01.772156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:01.771756    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:01.772156 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:01.771775    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:01.772156 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:01.771851    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:01.772156 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:01.771912    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:02.864203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:02.864171    2584 generic.go:356] "Generic (PLEG): container finished" podID="bd10017bb1a5437629e392659cbc55f4" containerID="dcc8d70c92310bcd177461f35d6786c8bc7a61e5b3281c3ba6bc23c5911b7dbb" exitCode=0
Jul 02 19:04:02.864624 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:02.864215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal" event={"ID":"bd10017bb1a5437629e392659cbc55f4","Type":"ContainerDied","Data":"dcc8d70c92310bcd177461f35d6786c8bc7a61e5b3281c3ba6bc23c5911b7dbb"}
Jul 02 19:04:03.000031 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:02.999843    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 19:04:03.771642 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.771611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:03.771816 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.771656    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:03.771816 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.771737    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:03.771816 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:03.771743    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:03.771986 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:03.771809    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:03.771986 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:03.771907    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:03.797155 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.797058    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:04:03.000028792Z","UUID":"f6048d51-f1cf-454d-942e-4abc860edec3","Handler":null,"Name":"","Endpoint":""}
Jul 02 19:04:03.798665 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.798648    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 19:04:03.798784 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.798685    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 19:04:03.869585 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.869545    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jdkdc" event={"ID":"c19b1252-7c1f-4ae2-86ad-47e5e4791628","Type":"ContainerStarted","Data":"7b8ab4e862ddfbaefc4968a57c50462e4c169ec1eef0b9741a7e4b0d1c03325c"}
Jul 02 19:04:03.871548 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.871523    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c" event={"ID":"7d8216ad-a1d3-46d7-9f79-0b36eb40443c","Type":"ContainerStarted","Data":"c92299541f8c359e7c7981dc69eb08c974376007ca1df3c7eb065aa4e3e39598"}
Jul 02 19:04:03.874741 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.874712    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"7a4095f975c6459dece3d0c152f5a1006413b2a144b024fbdf3858e0a42e7d85"}
Jul 02 19:04:03.874847 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.874779    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"4776035c883b6aa20de1fd2ba991341719e9fd1604126bae3b8ba1685a30ae50"}
Jul 02 19:04:03.874847 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.874830    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"5820d8d3d51a14a8eedd63ce091c8742a63c6f3891f733aa59473e00cafe6fc9"}
Jul 02 19:04:03.874966 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.874870    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"6f54e2841f6ecdb9a562c9efe76e5c32ab23aac31e056965cb079c62c8e1df40"}
Jul 02 19:04:03.874966 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.874914    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"3a260b1466cc795e8017d07b247499e831d03a7231bfd88d1feb9a377a49482f"}
Jul 02 19:04:03.874966 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.874960    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"0d1e87ff32c71177b8d44519e172f1d755869249b309d792a89e981953474567"}
Jul 02 19:04:03.876594 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.876574    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal" event={"ID":"bd10017bb1a5437629e392659cbc55f4","Type":"ContainerStarted","Data":"c87ff01f4753932be951fead5007fc8c8bbc73a8052a8fc406752791c964b057"}
Jul 02 19:04:03.889416 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:03.889373    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jdkdc" podStartSLOduration=2.99012045 podStartE2EDuration="22.889361557s" totalImagesPullingTime="19.899241107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:03.889176156 +0000 UTC m=+23.744107929" watchObservedRunningTime="2026-07-02 19:04:03.889361557 +0000 UTC m=+23.744293324"
Jul 02 19:04:04.880424 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:04.880389    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c" event={"ID":"7d8216ad-a1d3-46d7-9f79-0b36eb40443c","Type":"ContainerStarted","Data":"8a8384bede9e46bbb7b0971c950e8f532c86cf371ba7543aec21b6d4ba58927c"}
Jul 02 19:04:04.898038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:04.897992    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnl2c" podStartSLOduration=3.756352021 podStartE2EDuration="23.897977447s" totalImagesPullingTime="20.141625426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:04.89693609 +0000 UTC m=+24.751867861" watchObservedRunningTime="2026-07-02 19:04:04.897977447 +0000 UTC m=+24.752909234"
Jul 02 19:04:04.898244 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:04.898218    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-120.ec2.internal" podStartSLOduration=22.898208887 podStartE2EDuration="22.898208887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:03.905466498 +0000 UTC m=+23.760398267" watchObservedRunningTime="2026-07-02 19:04:04.898208887 +0000 UTC m=+24.753140658"
Jul 02 19:04:05.771172 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:05.771090    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:05.771318 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:05.771099    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:05.771318 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:05.771209    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:05.771318 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:05.771108    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:05.771318 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:05.771264    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:05.771504 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:05.771363    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:05.885410 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:05.885374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"e3b7edfb646b0767f98803f1508b74c2ac69d7dbdaf51b6180a677684b6954bc"}
Jul 02 19:04:07.771689 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:07.771652    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:07.772137 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:07.771787    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:07.772137 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:07.771799    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:07.772137 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:07.771803    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:07.772137 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:07.771910    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:07.772137 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:07.771985    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:08.896632 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.896307    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" event={"ID":"4d5ec31e-a746-4e9d-94c6-452d902740c8","Type":"ContainerStarted","Data":"b856f91c57e38c785591d9568a5a9f39cc2d64c84165352888f74ef4cd5c542e"}
Jul 02 19:04:08.897130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.896632    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:04:08.897130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.896692    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:04:08.897130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.896740    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:04:08.912047 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.912022    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:04:08.917781 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.917764    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:04:08.933284 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:08.933239    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2" podStartSLOduration=7.995416509 podStartE2EDuration="27.933226179s" totalImagesPullingTime="19.93780967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:08.931755714 +0000 UTC m=+28.786687507" watchObservedRunningTime="2026-07-02 19:04:08.933226179 +0000 UTC m=+28.788157948"
Jul 02 19:04:09.771651 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:09.771625    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:09.771790 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:09.771719    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:09.771790 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:09.771730    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:09.771851 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:09.771838    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:09.771893 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:09.771881    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:09.774562 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:09.772273    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:09.899617 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:09.899582    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3820882-8598-456f-9604-0306e0956be5" containerID="3a970ed7ea3efc91f6236adeded9053c92bd514a9d4211eceb5a85796969ee82" exitCode=0
Jul 02 19:04:09.899976 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:09.899653    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerDied","Data":"3a970ed7ea3efc91f6236adeded9053c92bd514a9d4211eceb5a85796969ee82"}
Jul 02 19:04:10.801153 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.800825    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2wgl5"]
Jul 02 19:04:10.801153 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.801130    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:10.801324 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:10.801292    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:10.801780 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.801757    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bj4fv"]
Jul 02 19:04:10.801891 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.801870    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:10.801995 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:10.801971    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:10.803543 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.803499    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hfqzn"]
Jul 02 19:04:10.803655 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.803577    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:10.803722 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:10.803659    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:10.904163 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:10.904139    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerStarted","Data":"0a2e36db1f7a6580bfee01565c0d240dd6f52f3d67df1305bb958a661d994755"}
Jul 02 19:04:11.908218 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:11.908137    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3820882-8598-456f-9604-0306e0956be5" containerID="0a2e36db1f7a6580bfee01565c0d240dd6f52f3d67df1305bb958a661d994755" exitCode=0
Jul 02 19:04:11.908218 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:11.908184    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerDied","Data":"0a2e36db1f7a6580bfee01565c0d240dd6f52f3d67df1305bb958a661d994755"}
Jul 02 19:04:12.772454 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:12.771880    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:12.772454 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:12.771896    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:12.772454 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:12.771896    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:12.772454 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:12.772003    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:12.772454 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:12.772076    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:12.772454 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:12.772132    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:12.912895 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:12.912863    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3820882-8598-456f-9604-0306e0956be5" containerID="164b7a128fb0e43422be133fecb838c5c4ce33795137264ca3f7b2376e8375b0" exitCode=0
Jul 02 19:04:12.913365 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:12.912907    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerDied","Data":"164b7a128fb0e43422be133fecb838c5c4ce33795137264ca3f7b2376e8375b0"}
Jul 02 19:04:14.341531 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:14.341490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:14.342043 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:14.341586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:14.342043 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.341590    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:04:14.342043 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.341640    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:04:14.342043 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.341673    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs podName:cb1ea074-9270-4ef6-997c-b177aaee9982 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:46.341652865 +0000 UTC m=+66.196584628 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs") pod "network-metrics-daemon-bj4fv" (UID: "cb1ea074-9270-4ef6-997c-b177aaee9982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:04:14.342043 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.341689    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret podName:631168e3-18d4-4ec6-9eb9-aea35e3f50d1 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:46.341681889 +0000 UTC m=+66.196613642 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret") pod "global-pull-secret-syncer-hfqzn" (UID: "631168e3-18d4-4ec6-9eb9-aea35e3f50d1") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:04:14.442296 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:14.442268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:14.442460 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.442398    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:04:14.442460 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.442416    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:04:14.442460 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.442424    2584 projected.go:196] Error preparing data for projected volume kube-api-access-pmhnf for pod openshift-network-diagnostics/network-check-target-2wgl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:04:14.442596 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.442471    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf podName:a23712d5-4f7f-4b11-b6dd-868d36cd75a3 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:46.442457281 +0000 UTC m=+66.297389029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmhnf" (UniqueName: "kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf") pod "network-check-target-2wgl5" (UID: "a23712d5-4f7f-4b11-b6dd-868d36cd75a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:04:14.771067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:14.770993    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:14.774038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:14.771256    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:14.774038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.771485    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfqzn" podUID="631168e3-18d4-4ec6-9eb9-aea35e3f50d1"
Jul 02 19:04:14.774038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.771616    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2wgl5" podUID="a23712d5-4f7f-4b11-b6dd-868d36cd75a3"
Jul 02 19:04:14.774038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:14.771658    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:14.774038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:14.773161    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bj4fv" podUID="cb1ea074-9270-4ef6-997c-b177aaee9982"
Jul 02 19:04:15.341776 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.341746    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-120.ec2.internal" event="NodeReady"
Jul 02 19:04:15.342142 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.341884    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 19:04:15.385655 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.385628    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8lk8m"]
Jul 02 19:04:15.397486 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.397459    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fzfd4"]
Jul 02 19:04:15.397615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.397573    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.400361 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.400318    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 19:04:15.400512 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.400369    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mg9mb\""
Jul 02 19:04:15.400512 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.400379    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 19:04:15.400512 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.400426    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 19:04:15.404038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.404008    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8lk8m"]
Jul 02 19:04:15.404130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.404108    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.406561 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.406544    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s5tjt\""
Jul 02 19:04:15.406752 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.406739    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 19:04:15.407074 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.407054    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 19:04:15.412370 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.412327    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fzfd4"]
Jul 02 19:04:15.550651 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.550573    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26d60aa7-42cc-4c6f-bfd4-8634586894b4-config-volume\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.550651 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.550608    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.550651 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.550634    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2rdb\" (UniqueName: \"kubernetes.io/projected/8cc512c5-ec30-4764-b576-8c8b7f183298-kube-api-access-g2rdb\") pod \"ingress-canary-8lk8m\" (UID: \"8cc512c5-ec30-4764-b576-8c8b7f183298\") " pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.550890 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.550737    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26d60aa7-42cc-4c6f-bfd4-8634586894b4-tmp-dir\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.550890 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.550775    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq9c2\" (UniqueName: \"kubernetes.io/projected/26d60aa7-42cc-4c6f-bfd4-8634586894b4-kube-api-access-bq9c2\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.550890 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.550801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cc512c5-ec30-4764-b576-8c8b7f183298-cert\") pod \"ingress-canary-8lk8m\" (UID: \"8cc512c5-ec30-4764-b576-8c8b7f183298\") " pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.651907 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.651872    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26d60aa7-42cc-4c6f-bfd4-8634586894b4-tmp-dir\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.652076 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.651922    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq9c2\" (UniqueName: \"kubernetes.io/projected/26d60aa7-42cc-4c6f-bfd4-8634586894b4-kube-api-access-bq9c2\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.652076 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.652040    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cc512c5-ec30-4764-b576-8c8b7f183298-cert\") pod \"ingress-canary-8lk8m\" (UID: \"8cc512c5-ec30-4764-b576-8c8b7f183298\") " pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.652179 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.652117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26d60aa7-42cc-4c6f-bfd4-8634586894b4-config-volume\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.652179 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.652142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.652179 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.652166    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rdb\" (UniqueName: \"kubernetes.io/projected/8cc512c5-ec30-4764-b576-8c8b7f183298-kube-api-access-g2rdb\") pod \"ingress-canary-8lk8m\" (UID: \"8cc512c5-ec30-4764-b576-8c8b7f183298\") " pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.652324 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:15.652233    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:04:15.652324 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.652248    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26d60aa7-42cc-4c6f-bfd4-8634586894b4-tmp-dir\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.652324 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:15.652292    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls podName:26d60aa7-42cc-4c6f-bfd4-8634586894b4 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:16.152275624 +0000 UTC m=+36.007207389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls") pod "dns-default-fzfd4" (UID: "26d60aa7-42cc-4c6f-bfd4-8634586894b4") : secret "dns-default-metrics-tls" not found
Jul 02 19:04:15.652651 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.652624    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26d60aa7-42cc-4c6f-bfd4-8634586894b4-config-volume\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.656312 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.656288    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cc512c5-ec30-4764-b576-8c8b7f183298-cert\") pod \"ingress-canary-8lk8m\" (UID: \"8cc512c5-ec30-4764-b576-8c8b7f183298\") " pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.661080 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.661055    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq9c2\" (UniqueName: \"kubernetes.io/projected/26d60aa7-42cc-4c6f-bfd4-8634586894b4-kube-api-access-bq9c2\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:15.661260 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.661245    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rdb\" (UniqueName: \"kubernetes.io/projected/8cc512c5-ec30-4764-b576-8c8b7f183298-kube-api-access-g2rdb\") pod \"ingress-canary-8lk8m\" (UID: \"8cc512c5-ec30-4764-b576-8c8b7f183298\") " pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.709049 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.709020    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8lk8m"
Jul 02 19:04:15.900550 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.900471    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8lk8m"]
Jul 02 19:04:15.904817 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:15.904786    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc512c5_ec30_4764_b576_8c8b7f183298.slice/crio-fc19670ac7b5dca428dccab9b84d60eb51abe85817bf0d05b6ed06d013492d74 WatchSource:0}: Error finding container fc19670ac7b5dca428dccab9b84d60eb51abe85817bf0d05b6ed06d013492d74: Status 404 returned error can't find the container with id fc19670ac7b5dca428dccab9b84d60eb51abe85817bf0d05b6ed06d013492d74
Jul 02 19:04:15.920555 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:15.920530    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8lk8m" event={"ID":"8cc512c5-ec30-4764-b576-8c8b7f183298","Type":"ContainerStarted","Data":"fc19670ac7b5dca428dccab9b84d60eb51abe85817bf0d05b6ed06d013492d74"}
Jul 02 19:04:16.155359 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.155305    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:16.155560 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:16.155486    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:04:16.155630 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:16.155563    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls podName:26d60aa7-42cc-4c6f-bfd4-8634586894b4 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:17.15554129 +0000 UTC m=+37.010473039 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls") pod "dns-default-fzfd4" (UID: "26d60aa7-42cc-4c6f-bfd4-8634586894b4") : secret "dns-default-metrics-tls" not found
Jul 02 19:04:16.771554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.771519    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:16.771554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.771560    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:16.772090 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.771518    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:16.774399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.774379    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 19:04:16.775363 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.775290    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 19:04:16.775363 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.775322    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 19:04:16.775363 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.775330    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6qb6\""
Jul 02 19:04:16.775593 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.775330    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9s6wg\""
Jul 02 19:04:16.775593 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:16.775386    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 19:04:17.162106 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:17.162066    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:17.162290 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:17.162239    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:04:17.162360 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:17.162331    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls podName:26d60aa7-42cc-4c6f-bfd4-8634586894b4 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:19.162313808 +0000 UTC m=+39.017245556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls") pod "dns-default-fzfd4" (UID: "26d60aa7-42cc-4c6f-bfd4-8634586894b4") : secret "dns-default-metrics-tls" not found
Jul 02 19:04:18.931406 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:18.931370    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3820882-8598-456f-9604-0306e0956be5" containerID="efe1e988447ca595ba22fa9882cbe7033a7230252ecadd968a95ecaadc0cdecd" exitCode=0
Jul 02 19:04:18.931802 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:18.931431    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerDied","Data":"efe1e988447ca595ba22fa9882cbe7033a7230252ecadd968a95ecaadc0cdecd"}
Jul 02 19:04:19.176665 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:19.176465    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:19.176835 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:19.176630    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:04:19.176835 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:19.176808    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls podName:26d60aa7-42cc-4c6f-bfd4-8634586894b4 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:23.176785876 +0000 UTC m=+43.031717641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls") pod "dns-default-fzfd4" (UID: "26d60aa7-42cc-4c6f-bfd4-8634586894b4") : secret "dns-default-metrics-tls" not found
Jul 02 19:04:19.938408 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:19.938376    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3820882-8598-456f-9604-0306e0956be5" containerID="0fa30e3251798c5ff76f19dc66a4e91c3aa8d7b1ac2f6baca7fbad14b99a7aaa" exitCode=0
Jul 02 19:04:19.938778 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:19.938424    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerDied","Data":"0fa30e3251798c5ff76f19dc66a4e91c3aa8d7b1ac2f6baca7fbad14b99a7aaa"}
Jul 02 19:04:20.942011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:20.941925    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8lk8m" event={"ID":"8cc512c5-ec30-4764-b576-8c8b7f183298","Type":"ContainerStarted","Data":"45f340619a992db6ec1ff56b497e4647beebc4ae5434f47ef00e5a2655b46209"}
Jul 02 19:04:20.945281 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:20.945258    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" event={"ID":"a3820882-8598-456f-9604-0306e0956be5","Type":"ContainerStarted","Data":"01691d7f0acb3eb020ab147dfd6431894ffc381c94f76db62744d86edfdd67ca"}
Jul 02 19:04:20.957436 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:20.957393    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8lk8m" podStartSLOduration=1.215172594 podStartE2EDuration="5.957379546s" totalImagesPullingTime="4.742206952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:20.956779297 +0000 UTC m=+40.811711068" watchObservedRunningTime="2026-07-02 19:04:20.957379546 +0000 UTC m=+40.812311294"
Jul 02 19:04:20.977305 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:20.977235    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m9qfc" podStartSLOduration=12.395286608 podStartE2EDuration="39.9772164s" totalImagesPullingTime="27.581929792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:20.975793345 +0000 UTC m=+40.830725116" watchObservedRunningTime="2026-07-02 19:04:20.9772164 +0000 UTC m=+40.832148165"
Jul 02 19:04:21.410609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.410579    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"]
Jul 02 19:04:21.436619 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.436593    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"
Jul 02 19:04:21.440032 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.440012    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 02 19:04:21.440175 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.440155    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hlfzt\""
Jul 02 19:04:21.443006 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.442805    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"]
Jul 02 19:04:21.445301 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.445283    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 02 19:04:21.491686 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.491659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvvg\" (UniqueName: \"kubernetes.io/projected/f6e90a60-b687-4089-88de-77a80339e0f1-kube-api-access-bgvvg\") pod \"migrator-d57d58cc9-4n2qw\" (UID: \"f6e90a60-b687-4089-88de-77a80339e0f1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"
Jul 02 19:04:21.592290 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.592257    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvvg\" (UniqueName: \"kubernetes.io/projected/f6e90a60-b687-4089-88de-77a80339e0f1-kube-api-access-bgvvg\") pod \"migrator-d57d58cc9-4n2qw\" (UID: \"f6e90a60-b687-4089-88de-77a80339e0f1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"
Jul 02 19:04:21.601492 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.601468    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvvg\" (UniqueName: \"kubernetes.io/projected/f6e90a60-b687-4089-88de-77a80339e0f1-kube-api-access-bgvvg\") pod \"migrator-d57d58cc9-4n2qw\" (UID: \"f6e90a60-b687-4089-88de-77a80339e0f1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"
Jul 02 19:04:21.731435 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.731358    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v6p5t"]
Jul 02 19:04:21.745065 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.745041    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"
Jul 02 19:04:21.756183 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.756160    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v6p5t"]
Jul 02 19:04:21.756289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.756269    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.758796 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.758771    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 19:04:21.758924 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.758850    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rhh92\""
Jul 02 19:04:21.758980 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.758772    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 19:04:21.759032 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.759011    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 19:04:21.759303 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.759198    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 19:04:21.793759 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.793729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.793911 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.793770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bae5299-39e9-4210-a777-a1ed57e4b596-data-volume\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.793911 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.793793    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzpcb\" (UniqueName: \"kubernetes.io/projected/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-api-access-gzpcb\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.793911 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.793871    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.794051 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.793953    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bae5299-39e9-4210-a777-a1ed57e4b596-crio-socket\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.885792 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.885613    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw"]
Jul 02 19:04:21.889522 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:21.889491    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e90a60_b687_4089_88de_77a80339e0f1.slice/crio-8072d25ae161b58d1de74b94e7a4a61d92123ce9968922d2f4b144e64f17e131 WatchSource:0}: Error finding container 8072d25ae161b58d1de74b94e7a4a61d92123ce9968922d2f4b144e64f17e131: Status 404 returned error can't find the container with id 8072d25ae161b58d1de74b94e7a4a61d92123ce9968922d2f4b144e64f17e131
Jul 02 19:04:21.895215 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.895299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895234    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bae5299-39e9-4210-a777-a1ed57e4b596-crio-socket\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.895299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.895299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895287    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bae5299-39e9-4210-a777-a1ed57e4b596-data-volume\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.895483 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895310    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzpcb\" (UniqueName: \"kubernetes.io/projected/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-api-access-gzpcb\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.895483 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:21.895328    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:21.895483 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:21.895391    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:04:21.895483 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:21.895424    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:22.395403879 +0000 UTC m=+42.250335636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:21.895483 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:21.895446    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:22.395436251 +0000 UTC m=+42.250368000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : configmap "kube-rbac-proxy" not found
Jul 02 19:04:21.895680 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895502    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bae5299-39e9-4210-a777-a1ed57e4b596-crio-socket\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.895680 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.895631    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bae5299-39e9-4210-a777-a1ed57e4b596-data-volume\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.903801 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.903783    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzpcb\" (UniqueName: \"kubernetes.io/projected/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-api-access-gzpcb\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:21.948485 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:21.948457    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw" event={"ID":"f6e90a60-b687-4089-88de-77a80339e0f1","Type":"ContainerStarted","Data":"8072d25ae161b58d1de74b94e7a4a61d92123ce9968922d2f4b144e64f17e131"}
Jul 02 19:04:22.398845 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:22.398802    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:22.399038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:22.398867    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:22.399038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:22.398944    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:04:22.399038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:22.398975    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:22.399038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:22.398996    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:23.398981939 +0000 UTC m=+43.253913687 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : configmap "kube-rbac-proxy" not found
Jul 02 19:04:22.399038 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:22.399037    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:23.399018981 +0000 UTC m=+43.253950762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:23.205370 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.205329    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:23.205731 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:23.205465    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:04:23.205731 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:23.205522    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls podName:26d60aa7-42cc-4c6f-bfd4-8634586894b4 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:31.205505144 +0000 UTC m=+51.060436906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls") pod "dns-default-fzfd4" (UID: "26d60aa7-42cc-4c6f-bfd4-8634586894b4") : secret "dns-default-metrics-tls" not found
Jul 02 19:04:23.407073 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.407040    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:23.407229 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.407117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:23.407229 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:23.407181    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:04:23.407229 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:23.407225    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:23.407365 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:23.407263    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:25.407240799 +0000 UTC m=+45.262172547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : configmap "kube-rbac-proxy" not found
Jul 02 19:04:23.407365 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:23.407284    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:25.407275046 +0000 UTC m=+45.262206796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:23.676833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.676798    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-m8fjc"]
Jul 02 19:04:23.695292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.695267    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-m8fjc"]
Jul 02 19:04:23.695460 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.695391    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.697534 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.697514    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 02 19:04:23.697645 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.697557    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 02 19:04:23.697875 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.697855    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 02 19:04:23.698563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.698541    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 02 19:04:23.698687 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.698652    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g54p6\""
Jul 02 19:04:23.710586 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.710566    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89c55bff-5f30-4042-990f-ac04b17a23fa-signing-key\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.710672 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.710607    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9lc\" (UniqueName: \"kubernetes.io/projected/89c55bff-5f30-4042-990f-ac04b17a23fa-kube-api-access-rb9lc\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.710672 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.710642    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89c55bff-5f30-4042-990f-ac04b17a23fa-signing-cabundle\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.811582 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.811559    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89c55bff-5f30-4042-990f-ac04b17a23fa-signing-key\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.811690 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.811599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9lc\" (UniqueName: \"kubernetes.io/projected/89c55bff-5f30-4042-990f-ac04b17a23fa-kube-api-access-rb9lc\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.811690 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.811629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89c55bff-5f30-4042-990f-ac04b17a23fa-signing-cabundle\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.812133 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.812117    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89c55bff-5f30-4042-990f-ac04b17a23fa-signing-cabundle\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.814103 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.814087    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89c55bff-5f30-4042-990f-ac04b17a23fa-signing-key\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.819249 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.819228    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9lc\" (UniqueName: \"kubernetes.io/projected/89c55bff-5f30-4042-990f-ac04b17a23fa-kube-api-access-rb9lc\") pod \"service-ca-758f599c5-m8fjc\" (UID: \"89c55bff-5f30-4042-990f-ac04b17a23fa\") " pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:23.956061 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:23.956038    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw" event={"ID":"f6e90a60-b687-4089-88de-77a80339e0f1","Type":"ContainerStarted","Data":"75fbdc01f88606a8dde3a9ea023b7d8e6ee7dc1aed724b538286655e7876b5e4"}
Jul 02 19:04:24.005118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:24.005096    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-m8fjc"
Jul 02 19:04:24.150543 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:24.150514    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-m8fjc"]
Jul 02 19:04:24.154207 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:24.154181    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c55bff_5f30_4042_990f_ac04b17a23fa.slice/crio-af4284f0d2702ff5a25711265a2fa454152007fcaec092818fc11153baa051ae WatchSource:0}: Error finding container af4284f0d2702ff5a25711265a2fa454152007fcaec092818fc11153baa051ae: Status 404 returned error can't find the container with id af4284f0d2702ff5a25711265a2fa454152007fcaec092818fc11153baa051ae
Jul 02 19:04:24.516588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:24.516562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9sz77_6079c29a-a61e-4487-96c2-831e5f272a17/dns-node-resolver/0.log"
Jul 02 19:04:24.959954 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:24.959755    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw" event={"ID":"f6e90a60-b687-4089-88de-77a80339e0f1","Type":"ContainerStarted","Data":"76fd7de2460b78ea615b5cd67611d58dc686f72a93ffadfb99392694456bcda8"}
Jul 02 19:04:24.960900 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:24.960875    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-m8fjc" event={"ID":"89c55bff-5f30-4042-990f-ac04b17a23fa","Type":"ContainerStarted","Data":"af4284f0d2702ff5a25711265a2fa454152007fcaec092818fc11153baa051ae"}
Jul 02 19:04:24.978802 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:24.978753    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4n2qw" podStartSLOduration=2.102582582 podStartE2EDuration="3.978739285s" totalImagesPullingTime="1.876156703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:24.977582021 +0000 UTC m=+44.832513784" watchObservedRunningTime="2026-07-02 19:04:24.978739285 +0000 UTC m=+44.833671049"
Jul 02 19:04:25.315605 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:25.315513    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sxpt2_f5963213-fcb7-4e5d-a018-152633bf39e8/node-ca/0.log"
Jul 02 19:04:25.428397 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:25.428318    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:25.428558 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:25.428423    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:25.428558 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:25.428538    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:04:25.428674 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:25.428570    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:25.428674 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:25.428608    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:29.428588749 +0000 UTC m=+49.283520514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : configmap "kube-rbac-proxy" not found
Jul 02 19:04:25.428674 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:25.428639    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:29.428619116 +0000 UTC m=+49.283550885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:26.114589 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:26.114562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8lk8m_8cc512c5-ec30-4764-b576-8c8b7f183298/serve-healthcheck-canary/0.log"
Jul 02 19:04:26.716904 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:26.716874    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-4n2qw_f6e90a60-b687-4089-88de-77a80339e0f1/migrator/0.log"
Jul 02 19:04:26.918795 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:26.918768    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-4n2qw_f6e90a60-b687-4089-88de-77a80339e0f1/graceful-termination/0.log"
Jul 02 19:04:27.969420 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:27.969384    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-m8fjc" event={"ID":"89c55bff-5f30-4042-990f-ac04b17a23fa","Type":"ContainerStarted","Data":"8c543e56c7122e08007d3327ff86586d39805f265c17d0de8cfa403e4244f63f"}
Jul 02 19:04:27.985106 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:27.985060    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-m8fjc" podStartSLOduration=2.215910475 podStartE2EDuration="4.985043111s" totalImagesPullingTime="2.769132636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:04:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:27.984010038 +0000 UTC m=+47.838941808" watchObservedRunningTime="2026-07-02 19:04:27.985043111 +0000 UTC m=+47.839974910"
Jul 02 19:04:29.461770 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:29.461736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:29.462168 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:29.461803    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:29.462168 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:29.461859    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:04:29.462168 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:29.461910    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:29.462168 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:29.461926    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:37.461910202 +0000 UTC m=+57.316841950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : configmap "kube-rbac-proxy" not found
Jul 02 19:04:29.462168 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:29.461951    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:37.461939683 +0000 UTC m=+57.316871431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:31.274246 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:31.274194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:31.274639 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:31.274371    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:04:31.274639 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:31.274464    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls podName:26d60aa7-42cc-4c6f-bfd4-8634586894b4 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:47.274445143 +0000 UTC m=+67.129376913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls") pod "dns-default-fzfd4" (UID: "26d60aa7-42cc-4c6f-bfd4-8634586894b4") : secret "dns-default-metrics-tls" not found
Jul 02 19:04:37.522513 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:37.522474    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:37.523081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:37.522553    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:37.523081 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:37.522625    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:37.523081 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:04:37.522691    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls podName:9bae5299-39e9-4210-a777-a1ed57e4b596 nodeName:}" failed. No retries permitted until 2026-07-02 19:04:53.522673887 +0000 UTC m=+73.377605638 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6p5t" (UID: "9bae5299-39e9-4210-a777-a1ed57e4b596") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:04:37.523081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:37.523063    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bae5299-39e9-4210-a777-a1ed57e4b596-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:40.925219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:40.925190    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2p9d2"
Jul 02 19:04:46.387170 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.387130    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:46.387170 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.387177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:46.389796 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.389777    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 19:04:46.389846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.389807    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 19:04:46.400772 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.400744    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb1ea074-9270-4ef6-997c-b177aaee9982-metrics-certs\") pod \"network-metrics-daemon-bj4fv\" (UID: \"cb1ea074-9270-4ef6-997c-b177aaee9982\") " pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:46.400890 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.400788    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/631168e3-18d4-4ec6-9eb9-aea35e3f50d1-original-pull-secret\") pod \"global-pull-secret-syncer-hfqzn\" (UID: \"631168e3-18d4-4ec6-9eb9-aea35e3f50d1\") " pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:46.482974 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.482934    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfqzn"
Jul 02 19:04:46.488516 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.488489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:46.490766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.490747    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 19:04:46.497528 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.497510    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9s6wg\""
Jul 02 19:04:46.500846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.500830    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 19:04:46.506655 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.506634    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bj4fv"
Jul 02 19:04:46.511750 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.511729    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmhnf\" (UniqueName: \"kubernetes.io/projected/a23712d5-4f7f-4b11-b6dd-868d36cd75a3-kube-api-access-pmhnf\") pod \"network-check-target-2wgl5\" (UID: \"a23712d5-4f7f-4b11-b6dd-868d36cd75a3\") " pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:46.613739 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.613710    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hfqzn"]
Jul 02 19:04:46.616901 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:46.616863    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631168e3_18d4_4ec6_9eb9_aea35e3f50d1.slice/crio-46a274ebe1fe4f6bfbd29d5d9dc9a5b4981e9394b9b40eedd6417230b9f37dcd WatchSource:0}: Error finding container 46a274ebe1fe4f6bfbd29d5d9dc9a5b4981e9394b9b40eedd6417230b9f37dcd: Status 404 returned error can't find the container with id 46a274ebe1fe4f6bfbd29d5d9dc9a5b4981e9394b9b40eedd6417230b9f37dcd
Jul 02 19:04:46.636177 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.636154    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bj4fv"]
Jul 02 19:04:46.638982 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:46.638933    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1ea074_9270_4ef6_997c_b177aaee9982.slice/crio-fcbf8892c0a8b4b1df29fb94acdee7c9b0adfbafc05d83f4b092955ad77e42c9 WatchSource:0}: Error finding container fcbf8892c0a8b4b1df29fb94acdee7c9b0adfbafc05d83f4b092955ad77e42c9: Status 404 returned error can't find the container with id fcbf8892c0a8b4b1df29fb94acdee7c9b0adfbafc05d83f4b092955ad77e42c9
Jul 02 19:04:46.792319 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.792289    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6qb6\""
Jul 02 19:04:46.800489 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.800472    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:46.923228 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:46.923150    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2wgl5"]
Jul 02 19:04:46.927287 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:46.927260    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23712d5_4f7f_4b11_b6dd_868d36cd75a3.slice/crio-807af847f5852f9dbff7b241cad233f58354528ed6d7370788aa30f8fdffaf65 WatchSource:0}: Error finding container 807af847f5852f9dbff7b241cad233f58354528ed6d7370788aa30f8fdffaf65: Status 404 returned error can't find the container with id 807af847f5852f9dbff7b241cad233f58354528ed6d7370788aa30f8fdffaf65
Jul 02 19:04:47.021898 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.021863    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bj4fv" event={"ID":"cb1ea074-9270-4ef6-997c-b177aaee9982","Type":"ContainerStarted","Data":"fcbf8892c0a8b4b1df29fb94acdee7c9b0adfbafc05d83f4b092955ad77e42c9"}
Jul 02 19:04:47.023219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.023191    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hfqzn" event={"ID":"631168e3-18d4-4ec6-9eb9-aea35e3f50d1","Type":"ContainerStarted","Data":"46a274ebe1fe4f6bfbd29d5d9dc9a5b4981e9394b9b40eedd6417230b9f37dcd"}
Jul 02 19:04:47.024384 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.024359    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2wgl5" event={"ID":"a23712d5-4f7f-4b11-b6dd-868d36cd75a3","Type":"ContainerStarted","Data":"807af847f5852f9dbff7b241cad233f58354528ed6d7370788aa30f8fdffaf65"}
Jul 02 19:04:47.296594 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.296438    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:47.299272 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.299249    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d60aa7-42cc-4c6f-bfd4-8634586894b4-metrics-tls\") pod \"dns-default-fzfd4\" (UID: \"26d60aa7-42cc-4c6f-bfd4-8634586894b4\") " pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:47.490680 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.490184    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"]
Jul 02 19:04:47.493531 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.493504    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"]
Jul 02 19:04:47.493729 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.493640    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.496439 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.496414    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 19:04:47.497008 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.496854    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.498277 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.498239    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 02 19:04:47.499096 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.499072    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 19:04:47.499282 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.499263    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-blwdn\""
Jul 02 19:04:47.500158 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.500141    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 02 19:04:47.501560 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.501000    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 02 19:04:47.501560 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.501308    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 02 19:04:47.503115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.501955    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 02 19:04:47.504536 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.504516    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 19:04:47.517141 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.517117    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"]
Jul 02 19:04:47.518309 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.518289    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s5tjt\""
Jul 02 19:04:47.524998 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.524976    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:47.540334 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.540271    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"]
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.598942    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-ca\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.598985    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-hub\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.599031    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5nb\" (UniqueName: \"kubernetes.io/projected/3fa6d0a1-4542-4991-8745-f1af2fea7401-kube-api-access-8p5nb\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.599063    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6fa92a08-a900-4961-b90b-e520e68f978c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c95765bfd-76b4h\" (UID: \"6fa92a08-a900-4961-b90b-e520e68f978c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.599121    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa6d0a1-4542-4991-8745-f1af2fea7401-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.599149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.599172    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.599685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.599208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcmjf\" (UniqueName: \"kubernetes.io/projected/6fa92a08-a900-4961-b90b-e520e68f978c-kube-api-access-qcmjf\") pod \"managed-serviceaccount-addon-agent-5c95765bfd-76b4h\" (UID: \"6fa92a08-a900-4961-b90b-e520e68f978c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.699891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-ca\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.699933    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-hub\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.699965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5nb\" (UniqueName: \"kubernetes.io/projected/3fa6d0a1-4542-4991-8745-f1af2fea7401-kube-api-access-8p5nb\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.699985    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6fa92a08-a900-4961-b90b-e520e68f978c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c95765bfd-76b4h\" (UID: \"6fa92a08-a900-4961-b90b-e520e68f978c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.700067    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa6d0a1-4542-4991-8745-f1af2fea7401-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.700097    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.700122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.700570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.700151    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcmjf\" (UniqueName: \"kubernetes.io/projected/6fa92a08-a900-4961-b90b-e520e68f978c-kube-api-access-qcmjf\") pod \"managed-serviceaccount-addon-agent-5c95765bfd-76b4h\" (UID: \"6fa92a08-a900-4961-b90b-e520e68f978c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.701761 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.701705    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa6d0a1-4542-4991-8745-f1af2fea7401-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.704139 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.704110    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-ca\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.705607 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.705560    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6fa92a08-a900-4961-b90b-e520e68f978c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c95765bfd-76b4h\" (UID: \"6fa92a08-a900-4961-b90b-e520e68f978c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.705718 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.705692    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.706328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.706306    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-hub\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.707058 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.707013    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fa6d0a1-4542-4991-8745-f1af2fea7401-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.713723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.713618    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fzfd4"]
Jul 02 19:04:47.716755 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.716659    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcmjf\" (UniqueName: \"kubernetes.io/projected/6fa92a08-a900-4961-b90b-e520e68f978c-kube-api-access-qcmjf\") pod \"managed-serviceaccount-addon-agent-5c95765bfd-76b4h\" (UID: \"6fa92a08-a900-4961-b90b-e520e68f978c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.716903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.716884    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5nb\" (UniqueName: \"kubernetes.io/projected/3fa6d0a1-4542-4991-8745-f1af2fea7401-kube-api-access-8p5nb\") pod \"cluster-proxy-proxy-agent-76cd8757bf-7bkdx\" (UID: \"3fa6d0a1-4542-4991-8745-f1af2fea7401\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.829468 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.829429    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"
Jul 02 19:04:47.838515 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:47.838301    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"
Jul 02 19:04:47.940962 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:47.940922    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d60aa7_42cc_4c6f_bfd4_8634586894b4.slice/crio-0749d63e74ca2dbda1fa50dcb3b06ef0d097bd1c9fb39c8f97d22a003f9b7d21 WatchSource:0}: Error finding container 0749d63e74ca2dbda1fa50dcb3b06ef0d097bd1c9fb39c8f97d22a003f9b7d21: Status 404 returned error can't find the container with id 0749d63e74ca2dbda1fa50dcb3b06ef0d097bd1c9fb39c8f97d22a003f9b7d21
Jul 02 19:04:48.032108 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:48.032023    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fzfd4" event={"ID":"26d60aa7-42cc-4c6f-bfd4-8634586894b4","Type":"ContainerStarted","Data":"0749d63e74ca2dbda1fa50dcb3b06ef0d097bd1c9fb39c8f97d22a003f9b7d21"}
Jul 02 19:04:48.096750 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:48.096614    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h"]
Jul 02 19:04:48.106488 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:48.106452    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fa92a08_a900_4961_b90b_e520e68f978c.slice/crio-92fb7a2d0a1bfc99ca2519345f59a03edc24d5ae4efdf0422efb92d89cf8a37c WatchSource:0}: Error finding container 92fb7a2d0a1bfc99ca2519345f59a03edc24d5ae4efdf0422efb92d89cf8a37c: Status 404 returned error can't find the container with id 92fb7a2d0a1bfc99ca2519345f59a03edc24d5ae4efdf0422efb92d89cf8a37c
Jul 02 19:04:48.127412 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:48.127334    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx"]
Jul 02 19:04:48.132290 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:48.132258    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fa6d0a1_4542_4991_8745_f1af2fea7401.slice/crio-cdb96cc0d0b8e39b7fbc0d71dd10f3e3987f1563421d8f0375484d2a67a99a17 WatchSource:0}: Error finding container cdb96cc0d0b8e39b7fbc0d71dd10f3e3987f1563421d8f0375484d2a67a99a17: Status 404 returned error can't find the container with id cdb96cc0d0b8e39b7fbc0d71dd10f3e3987f1563421d8f0375484d2a67a99a17
Jul 02 19:04:49.040517 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:49.039981    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bj4fv" event={"ID":"cb1ea074-9270-4ef6-997c-b177aaee9982","Type":"ContainerStarted","Data":"fd073f772d377dbc6203e77c6552d192b9695f9817ad3c65e8b6e226bfdbafe4"}
Jul 02 19:04:49.040517 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:49.040040    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bj4fv" event={"ID":"cb1ea074-9270-4ef6-997c-b177aaee9982","Type":"ContainerStarted","Data":"973580d906b319c3de579282af3ed64c900b8e254f770005cf5aa6450db1a4d0"}
Jul 02 19:04:49.041899 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:49.041868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx" event={"ID":"3fa6d0a1-4542-4991-8745-f1af2fea7401","Type":"ContainerStarted","Data":"cdb96cc0d0b8e39b7fbc0d71dd10f3e3987f1563421d8f0375484d2a67a99a17"}
Jul 02 19:04:49.043704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:49.043664    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h" event={"ID":"6fa92a08-a900-4961-b90b-e520e68f978c","Type":"ContainerStarted","Data":"92fb7a2d0a1bfc99ca2519345f59a03edc24d5ae4efdf0422efb92d89cf8a37c"}
Jul 02 19:04:49.059401 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:49.058793    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bj4fv" podStartSLOduration=66.708659777 podStartE2EDuration="1m8.058772693s" totalImagesPullingTime="1.350112916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:49.057628899 +0000 UTC m=+68.912560670" watchObservedRunningTime="2026-07-02 19:04:49.058772693 +0000 UTC m=+68.913704465"
Jul 02 19:04:53.549988 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:53.549952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:53.552256 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:53.552223    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bae5299-39e9-4210-a777-a1ed57e4b596-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6p5t\" (UID: \"9bae5299-39e9-4210-a777-a1ed57e4b596\") " pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:53.570419 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:53.570394    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rhh92\""
Jul 02 19:04:53.578238 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:53.578217    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v6p5t"
Jul 02 19:04:54.553898 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:54.553820    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v6p5t"]
Jul 02 19:04:54.559176 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:04:54.559147    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bae5299_39e9_4210_a777_a1ed57e4b596.slice/crio-3f9c54e84da1d96ef77d487e01b2fa384fb7f772c1ab7d9777439708d49b0c75 WatchSource:0}: Error finding container 3f9c54e84da1d96ef77d487e01b2fa384fb7f772c1ab7d9777439708d49b0c75: Status 404 returned error can't find the container with id 3f9c54e84da1d96ef77d487e01b2fa384fb7f772c1ab7d9777439708d49b0c75
Jul 02 19:04:55.074662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.073940    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fzfd4" event={"ID":"26d60aa7-42cc-4c6f-bfd4-8634586894b4","Type":"ContainerStarted","Data":"6c199acf40f320ad0f8db2882371cf37cd4172db54cbea25c8a6d31fe8e16ab1"}
Jul 02 19:04:55.074662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.074030    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fzfd4" event={"ID":"26d60aa7-42cc-4c6f-bfd4-8634586894b4","Type":"ContainerStarted","Data":"5fe567be6e6e4d505d713c991dd2332f0453da521e02e644db40f272f679bb6d"}
Jul 02 19:04:55.074662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.074283    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:04:55.077022 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.076720    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h" event={"ID":"6fa92a08-a900-4961-b90b-e520e68f978c","Type":"ContainerStarted","Data":"256fdd0325ce282a6256fc39e16bc33f60b27528e01ab059154da07ebca4c03a"}
Jul 02 19:04:55.080050 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.079599    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2wgl5" event={"ID":"a23712d5-4f7f-4b11-b6dd-868d36cd75a3","Type":"ContainerStarted","Data":"e03aa73dc8ab81749cb770dbd11cac3ade246d0e70de5bbcae4e7cbce4a2b559"}
Jul 02 19:04:55.080050 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.079994    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:04:55.081877 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.081831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hfqzn" event={"ID":"631168e3-18d4-4ec6-9eb9-aea35e3f50d1","Type":"ContainerStarted","Data":"ba6a1aede21768c6116284e910ed87ae9e9cb59295e01106b592675e39dcc410"}
Jul 02 19:04:55.083279 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.083259    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6p5t" event={"ID":"9bae5299-39e9-4210-a777-a1ed57e4b596","Type":"ContainerStarted","Data":"3f9c54e84da1d96ef77d487e01b2fa384fb7f772c1ab7d9777439708d49b0c75"}
Jul 02 19:04:55.093596 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.093543    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fzfd4" podStartSLOduration=33.617632348 podStartE2EDuration="40.093523436s" totalImagesPullingTime="6.475891088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:55.092063638 +0000 UTC m=+74.946995440" watchObservedRunningTime="2026-07-02 19:04:55.093523436 +0000 UTC m=+74.948455206"
Jul 02 19:04:55.108418 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.107386    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hfqzn" podStartSLOduration=66.305881654 podStartE2EDuration="1m14.107370095s" totalImagesPullingTime="7.801488441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:55.106787601 +0000 UTC m=+74.961719396" watchObservedRunningTime="2026-07-02 19:04:55.107370095 +0000 UTC m=+74.962301856"
Jul 02 19:04:55.137685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.137551    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2wgl5" podStartSLOduration=66.648457782 podStartE2EDuration="1m14.137532895s" totalImagesPullingTime="7.489075113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:55.136745452 +0000 UTC m=+74.991677223" watchObservedRunningTime="2026-07-02 19:04:55.137532895 +0000 UTC m=+74.992464663"
Jul 02 19:04:55.138624 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:55.138577    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c95765bfd-76b4h" podStartSLOduration=1.8222023790000001 podStartE2EDuration="8.138563618s" totalImagesPullingTime="6.316361239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:04:55.122629944 +0000 UTC m=+74.977561716" watchObservedRunningTime="2026-07-02 19:04:55.138563618 +0000 UTC m=+74.993495391"
Jul 02 19:04:59.098757 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:59.098720    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6p5t" event={"ID":"9bae5299-39e9-4210-a777-a1ed57e4b596","Type":"ContainerStarted","Data":"b99f658bcca0ecabf65077026559abc7f6989754a2099fcbcce281858e63cfe3"}
Jul 02 19:04:59.099176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:04:59.098822    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6p5t" event={"ID":"9bae5299-39e9-4210-a777-a1ed57e4b596","Type":"ContainerStarted","Data":"795128dc9839294cb891c19ebbdc8839ea8b791a4fff9258d3635d1a1cb726d1"}
Jul 02 19:05:01.310099 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.310042    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r4lv9"]
Jul 02 19:05:01.321715 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.321690    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.324533 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.324506    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 19:05:01.325154 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.325130    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gbb56\""
Jul 02 19:05:01.325679 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.325655    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 19:05:01.325796 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.325703    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 19:05:01.325925 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.325912    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 19:05:01.326496 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.326364    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 19:05:01.326708 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.326564    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 19:05:01.407563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407531    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-wtmp\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-textfile\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407614    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6d210e0-c015-494e-bcd4-92d75be978e9-metrics-client-ca\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9vgb\" (UniqueName: \"kubernetes.io/projected/b6d210e0-c015-494e-bcd4-92d75be978e9-kube-api-access-n9vgb\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407699    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407836    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-sys\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407860    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-root\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.407952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.407884    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-tls\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508389    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6d210e0-c015-494e-bcd4-92d75be978e9-metrics-client-ca\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508519 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508428    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9vgb\" (UniqueName: \"kubernetes.io/projected/b6d210e0-c015-494e-bcd4-92d75be978e9-kube-api-access-n9vgb\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508519 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508462    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508519 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508531    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-sys\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-root\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-tls\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-wtmp\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508858 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-textfile\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508858 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508829    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-root\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508959 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508888    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-sys\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.508959 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508938    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-textfile\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.509057 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.508975    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6d210e0-c015-494e-bcd4-92d75be978e9-metrics-client-ca\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.509115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.509103    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-wtmp\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.509167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.509123    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.511102 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.511082    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.511439 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.511425    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6d210e0-c015-494e-bcd4-92d75be978e9-node-exporter-tls\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.516377 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.516336    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9vgb\" (UniqueName: \"kubernetes.io/projected/b6d210e0-c015-494e-bcd4-92d75be978e9-kube-api-access-n9vgb\") pod \"node-exporter-r4lv9\" (UID: \"b6d210e0-c015-494e-bcd4-92d75be978e9\") " pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.633311 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:01.633280    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r4lv9"
Jul 02 19:05:01.644081 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:05:01.644051    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d210e0_c015_494e_bcd4_92d75be978e9.slice/crio-19c08ef30a3bbff99751159533a6a6e7a49227233f64d0c5625b825b7226d604 WatchSource:0}: Error finding container 19c08ef30a3bbff99751159533a6a6e7a49227233f64d0c5625b825b7226d604: Status 404 returned error can't find the container with id 19c08ef30a3bbff99751159533a6a6e7a49227233f64d0c5625b825b7226d604
Jul 02 19:05:02.111658 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:02.111543    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4lv9" event={"ID":"b6d210e0-c015-494e-bcd4-92d75be978e9","Type":"ContainerStarted","Data":"19c08ef30a3bbff99751159533a6a6e7a49227233f64d0c5625b825b7226d604"}
Jul 02 19:05:02.113928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:02.113891    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6p5t" event={"ID":"9bae5299-39e9-4210-a777-a1ed57e4b596","Type":"ContainerStarted","Data":"bd8fb39e264916681490a4b182c241d94172ce1ffd78861614a3aeec12a81de0"}
Jul 02 19:05:02.131413 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:02.131360    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v6p5t" podStartSLOduration=34.316005694 podStartE2EDuration="41.131298894s" totalImagesPullingTime="6.8152932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:04:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:02.131039323 +0000 UTC m=+81.985971093" watchObservedRunningTime="2026-07-02 19:05:02.131298894 +0000 UTC m=+81.986230666"
Jul 02 19:05:03.118797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:03.118764    2584 generic.go:356] "Generic (PLEG): container finished" podID="b6d210e0-c015-494e-bcd4-92d75be978e9" containerID="4ead62c5947e888d609867e764378c796e71e4d51dc3f4e144dea35815aab41e" exitCode=0
Jul 02 19:05:03.119269 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:03.118850    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4lv9" event={"ID":"b6d210e0-c015-494e-bcd4-92d75be978e9","Type":"ContainerDied","Data":"4ead62c5947e888d609867e764378c796e71e4d51dc3f4e144dea35815aab41e"}
Jul 02 19:05:04.125481 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:04.125439    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4lv9" event={"ID":"b6d210e0-c015-494e-bcd4-92d75be978e9","Type":"ContainerStarted","Data":"62f06c4b6bd2483c2d7ec612d00791bddc4a3b1db28bafc5930865fcbacd7965"}
Jul 02 19:05:04.125915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:04.125506    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4lv9" event={"ID":"b6d210e0-c015-494e-bcd4-92d75be978e9","Type":"ContainerStarted","Data":"a7497ab7eb00b581119e5b737793eb400cb3f5f9d3ce206ff68b4cb49cfb4324"}
Jul 02 19:05:04.147023 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:04.146961    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r4lv9" podStartSLOduration=1.769527728 podStartE2EDuration="3.146940932s" totalImagesPullingTime="1.300911026s" totalInitContainerRuntime="76.502178ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:04.145602623 +0000 UTC m=+84.000534397" watchObservedRunningTime="2026-07-02 19:05:04.146940932 +0000 UTC m=+84.001872702"
Jul 02 19:05:05.088198 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:05.088171    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fzfd4"
Jul 02 19:05:05.133717 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:05.133666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx" event={"ID":"3fa6d0a1-4542-4991-8745-f1af2fea7401","Type":"ContainerStarted","Data":"f82cf6f80d08a8ae03f46267ffa1b814e79804684758be5b53c932c5d90331ad"}
Jul 02 19:05:05.134219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:05.133735    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx" event={"ID":"3fa6d0a1-4542-4991-8745-f1af2fea7401","Type":"ContainerStarted","Data":"5872748cacda80384f0a0d82ee81de51dd6884ca33eae1c6c831808d312f5668"}
Jul 02 19:05:05.134219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:05.133755    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx" event={"ID":"3fa6d0a1-4542-4991-8745-f1af2fea7401","Type":"ContainerStarted","Data":"a983c2ef7194f16fe0b532e472d7063db32c466ca765e81cc3e2b26c1852c99d"}
Jul 02 19:05:05.152723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:05.152674    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cd8757bf-7bkdx" podStartSLOduration=1.6898078079999999 podStartE2EDuration="18.15265431s" totalImagesPullingTime="16.462846502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:05.151820295 +0000 UTC m=+85.006752065" watchObservedRunningTime="2026-07-02 19:05:05.15265431 +0000 UTC m=+85.007586083"
Jul 02 19:05:06.025063 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.025024    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"]
Jul 02 19:05:06.028199 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.028179    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.030608 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.030587    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 02 19:05:06.030706 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.030626    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5422g\""
Jul 02 19:05:06.036960 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.036933    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"]
Jul 02 19:05:06.145941 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.145916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qssrt\" (UniqueName: \"kubernetes.io/projected/b98df59c-b29e-4255-85c1-5bcdfe556aa6-kube-api-access-qssrt\") pod \"monitoring-plugin-d5dcff7cf-f2dkn\" (UID: \"b98df59c-b29e-4255-85c1-5bcdfe556aa6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.145941 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.145946    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b98df59c-b29e-4255-85c1-5bcdfe556aa6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-f2dkn\" (UID: \"b98df59c-b29e-4255-85c1-5bcdfe556aa6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.246329 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.246291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qssrt\" (UniqueName: \"kubernetes.io/projected/b98df59c-b29e-4255-85c1-5bcdfe556aa6-kube-api-access-qssrt\") pod \"monitoring-plugin-d5dcff7cf-f2dkn\" (UID: \"b98df59c-b29e-4255-85c1-5bcdfe556aa6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.246518 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.246403    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b98df59c-b29e-4255-85c1-5bcdfe556aa6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-f2dkn\" (UID: \"b98df59c-b29e-4255-85c1-5bcdfe556aa6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.249011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.248979    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b98df59c-b29e-4255-85c1-5bcdfe556aa6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-f2dkn\" (UID: \"b98df59c-b29e-4255-85c1-5bcdfe556aa6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.253395 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.253375    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qssrt\" (UniqueName: \"kubernetes.io/projected/b98df59c-b29e-4255-85c1-5bcdfe556aa6-kube-api-access-qssrt\") pod \"monitoring-plugin-d5dcff7cf-f2dkn\" (UID: \"b98df59c-b29e-4255-85c1-5bcdfe556aa6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.337530 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.337504    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:06.458615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:06.458584    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"]
Jul 02 19:05:06.462012 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:05:06.461988    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98df59c_b29e_4255_85c1_5bcdfe556aa6.slice/crio-e8d774906b6112a73a54abc5a5f239ab5173d550192c2d98db5d11d8478a9787 WatchSource:0}: Error finding container e8d774906b6112a73a54abc5a5f239ab5173d550192c2d98db5d11d8478a9787: Status 404 returned error can't find the container with id e8d774906b6112a73a54abc5a5f239ab5173d550192c2d98db5d11d8478a9787
Jul 02 19:05:07.142831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:07.142796    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn" event={"ID":"b98df59c-b29e-4255-85c1-5bcdfe556aa6","Type":"ContainerStarted","Data":"e8d774906b6112a73a54abc5a5f239ab5173d550192c2d98db5d11d8478a9787"}
Jul 02 19:05:08.148225 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:08.148141    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn" event={"ID":"b98df59c-b29e-4255-85c1-5bcdfe556aa6","Type":"ContainerStarted","Data":"ef8514ef32b7fc24e8417e8121560ab07ad4962f15d5c51c7d3151fc72b353a4"}
Jul 02 19:05:08.148603 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:08.148301    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:08.153000 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:08.152977    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn"
Jul 02 19:05:08.163801 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:08.163762    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-f2dkn" podStartSLOduration=0.774650599 podStartE2EDuration="2.163749039s" totalImagesPullingTime="1.38909844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:08.163011408 +0000 UTC m=+88.017943180" watchObservedRunningTime="2026-07-02 19:05:08.163749039 +0000 UTC m=+88.018680809"
Jul 02 19:05:27.093396 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:27.093364    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2wgl5"
Jul 02 19:05:42.840951 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:05:42.840906    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8lk8m_8cc512c5-ec30-4764-b576-8c8b7f183298/serve-healthcheck-canary/0.log"
Jul 02 19:08:05.601124 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.601047    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4zk58"]
Jul 02 19:08:05.603164 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.603146    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.606790 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.606767    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 02 19:08:05.607767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.607750    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 02 19:08:05.607832 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.607790    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vnsj9\""
Jul 02 19:08:05.622086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.622066    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4zk58"]
Jul 02 19:08:05.751577 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.751544    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69w42\" (UniqueName: \"kubernetes.io/projected/f616d8a0-c93a-427a-937a-ce5ed53b4b6b-kube-api-access-69w42\") pod \"cert-manager-webhook-6486ffd997-4zk58\" (UID: \"f616d8a0-c93a-427a-937a-ce5ed53b4b6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.751751 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.751606    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f616d8a0-c93a-427a-937a-ce5ed53b4b6b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4zk58\" (UID: \"f616d8a0-c93a-427a-937a-ce5ed53b4b6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.852708 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.852626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f616d8a0-c93a-427a-937a-ce5ed53b4b6b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4zk58\" (UID: \"f616d8a0-c93a-427a-937a-ce5ed53b4b6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.852708 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.852674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69w42\" (UniqueName: \"kubernetes.io/projected/f616d8a0-c93a-427a-937a-ce5ed53b4b6b-kube-api-access-69w42\") pod \"cert-manager-webhook-6486ffd997-4zk58\" (UID: \"f616d8a0-c93a-427a-937a-ce5ed53b4b6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.872666 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.872640    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f616d8a0-c93a-427a-937a-ce5ed53b4b6b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4zk58\" (UID: \"f616d8a0-c93a-427a-937a-ce5ed53b4b6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.872806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.872680    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69w42\" (UniqueName: \"kubernetes.io/projected/f616d8a0-c93a-427a-937a-ce5ed53b4b6b-kube-api-access-69w42\") pod \"cert-manager-webhook-6486ffd997-4zk58\" (UID: \"f616d8a0-c93a-427a-937a-ce5ed53b4b6b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:05.911539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:05.911517    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:06.053745 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:06.053718    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4zk58"]
Jul 02 19:08:06.054790 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:08:06.054765    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf616d8a0_c93a_427a_937a_ce5ed53b4b6b.slice/crio-7e795e88df7d15d5b453ecc2d5c22e1f80da205d6f3055425301d6956d0877dc WatchSource:0}: Error finding container 7e795e88df7d15d5b453ecc2d5c22e1f80da205d6f3055425301d6956d0877dc: Status 404 returned error can't find the container with id 7e795e88df7d15d5b453ecc2d5c22e1f80da205d6f3055425301d6956d0877dc
Jul 02 19:08:06.733397 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:06.733359    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58" event={"ID":"f616d8a0-c93a-427a-937a-ce5ed53b4b6b","Type":"ContainerStarted","Data":"7e795e88df7d15d5b453ecc2d5c22e1f80da205d6f3055425301d6956d0877dc"}
Jul 02 19:08:09.746022 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:09.745980    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58" event={"ID":"f616d8a0-c93a-427a-937a-ce5ed53b4b6b","Type":"ContainerStarted","Data":"225c29ca54d3380c989c182cf2a41134c0b808a2ac547b2b0d25198d89befa2b"}
Jul 02 19:08:09.746434 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:09.746068    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:09.763749 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:09.763695    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58" podStartSLOduration=1.208116141 podStartE2EDuration="4.763678372s" totalImagesPullingTime="3.555562231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:09.763466194 +0000 UTC m=+269.618397985" watchObservedRunningTime="2026-07-02 19:08:09.763678372 +0000 UTC m=+269.618610145"
Jul 02 19:08:15.751267 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:15.751240    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4zk58"
Jul 02 19:08:25.962317 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:25.962280    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ndkfs"]
Jul 02 19:08:25.964321 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:25.964305    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:25.966604 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:25.966585    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-58mcp\""
Jul 02 19:08:25.973078 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:25.973056    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ndkfs"]
Jul 02 19:08:25.995833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:25.995806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94085662-ce85-44e4-a889-39110873a60e-bound-sa-token\") pod \"cert-manager-55b4d954d5-ndkfs\" (UID: \"94085662-ce85-44e4-a889-39110873a60e\") " pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:25.995944 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:25.995845    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqlxl\" (UniqueName: \"kubernetes.io/projected/94085662-ce85-44e4-a889-39110873a60e-kube-api-access-jqlxl\") pod \"cert-manager-55b4d954d5-ndkfs\" (UID: \"94085662-ce85-44e4-a889-39110873a60e\") " pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:26.096509 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.096480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94085662-ce85-44e4-a889-39110873a60e-bound-sa-token\") pod \"cert-manager-55b4d954d5-ndkfs\" (UID: \"94085662-ce85-44e4-a889-39110873a60e\") " pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:26.096665 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.096519    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqlxl\" (UniqueName: \"kubernetes.io/projected/94085662-ce85-44e4-a889-39110873a60e-kube-api-access-jqlxl\") pod \"cert-manager-55b4d954d5-ndkfs\" (UID: \"94085662-ce85-44e4-a889-39110873a60e\") " pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:26.105992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.105969    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqlxl\" (UniqueName: \"kubernetes.io/projected/94085662-ce85-44e4-a889-39110873a60e-kube-api-access-jqlxl\") pod \"cert-manager-55b4d954d5-ndkfs\" (UID: \"94085662-ce85-44e4-a889-39110873a60e\") " pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:26.106208 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.106190    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94085662-ce85-44e4-a889-39110873a60e-bound-sa-token\") pod \"cert-manager-55b4d954d5-ndkfs\" (UID: \"94085662-ce85-44e4-a889-39110873a60e\") " pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:26.273255 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.273170    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ndkfs"
Jul 02 19:08:26.403008 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.402977    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ndkfs"]
Jul 02 19:08:26.405901 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:08:26.405865    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94085662_ce85_44e4_a889_39110873a60e.slice/crio-ec1667194eb3f58930b843069ca3c51e183650ad0777516540c5f1c7e2fbe9e9 WatchSource:0}: Error finding container ec1667194eb3f58930b843069ca3c51e183650ad0777516540c5f1c7e2fbe9e9: Status 404 returned error can't find the container with id ec1667194eb3f58930b843069ca3c51e183650ad0777516540c5f1c7e2fbe9e9
Jul 02 19:08:26.801301 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.801267    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ndkfs" event={"ID":"94085662-ce85-44e4-a889-39110873a60e","Type":"ContainerStarted","Data":"794cea14026074af03bc8d3feba00375ec056d9b5968c6b8a8ee76d5da4fe14d"}
Jul 02 19:08:26.801301 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.801306    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ndkfs" event={"ID":"94085662-ce85-44e4-a889-39110873a60e","Type":"ContainerStarted","Data":"ec1667194eb3f58930b843069ca3c51e183650ad0777516540c5f1c7e2fbe9e9"}
Jul 02 19:08:26.818125 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:26.818078    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ndkfs" podStartSLOduration=1.818063961 podStartE2EDuration="1.818063961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:26.817279353 +0000 UTC m=+286.672211327" watchObservedRunningTime="2026-07-02 19:08:26.818063961 +0000 UTC m=+286.672995731"
Jul 02 19:08:40.567746 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:08:40.567716    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 19:09:05.767646 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.767608    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"]
Jul 02 19:09:05.769583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.769568    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.772398 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.772378    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 02 19:09:05.773062 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.773042    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 02 19:09:05.773146 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.773048    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 02 19:09:05.773146 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.773049    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 02 19:09:05.773146 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.773123    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tnjqd\""
Jul 02 19:09:05.786112 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.786091    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"]
Jul 02 19:09:05.883292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.883261    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8954e82e-6c64-4c92-814d-702dc9889237-manifests\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.883451 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.883302    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8954e82e-6c64-4c92-814d-702dc9889237-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.883451 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.883325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8954e82e-6c64-4c92-814d-702dc9889237-webhook-cert\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.883451 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.883416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69g7h\" (UniqueName: \"kubernetes.io/projected/8954e82e-6c64-4c92-814d-702dc9889237-kube-api-access-69g7h\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.883589 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.883457    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8954e82e-6c64-4c92-814d-702dc9889237-tmp\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.984267 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.984231    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8954e82e-6c64-4c92-814d-702dc9889237-manifests\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.984471 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.984273    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8954e82e-6c64-4c92-814d-702dc9889237-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.984471 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.984296    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8954e82e-6c64-4c92-814d-702dc9889237-webhook-cert\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.984471 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.984359    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69g7h\" (UniqueName: \"kubernetes.io/projected/8954e82e-6c64-4c92-814d-702dc9889237-kube-api-access-69g7h\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.984471 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.984404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8954e82e-6c64-4c92-814d-702dc9889237-tmp\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.984688 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.984672    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8954e82e-6c64-4c92-814d-702dc9889237-manifests\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.986645 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.986620    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8954e82e-6c64-4c92-814d-702dc9889237-tmp\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.986776 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.986758    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8954e82e-6c64-4c92-814d-702dc9889237-webhook-cert\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.986879 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.986864    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8954e82e-6c64-4c92-814d-702dc9889237-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:05.993490 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:05.993462    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69g7h\" (UniqueName: \"kubernetes.io/projected/8954e82e-6c64-4c92-814d-702dc9889237-kube-api-access-69g7h\") pod \"opendatahub-operator-controller-manager-cb89c69-4rjpk\" (UID: \"8954e82e-6c64-4c92-814d-702dc9889237\") " pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:06.079656 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:06.079569    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:06.208697 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:06.208660    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"]
Jul 02 19:09:06.211737 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:09:06.211709    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8954e82e_6c64_4c92_814d_702dc9889237.slice/crio-a699139f9d32e592c3334aa564eeb5165df68c30c53af254d1e1fa19b1a5b5b6 WatchSource:0}: Error finding container a699139f9d32e592c3334aa564eeb5165df68c30c53af254d1e1fa19b1a5b5b6: Status 404 returned error can't find the container with id a699139f9d32e592c3334aa564eeb5165df68c30c53af254d1e1fa19b1a5b5b6
Jul 02 19:09:06.213499 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:06.213480    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:09:06.942226 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:06.942178    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk" event={"ID":"8954e82e-6c64-4c92-814d-702dc9889237","Type":"ContainerStarted","Data":"a699139f9d32e592c3334aa564eeb5165df68c30c53af254d1e1fa19b1a5b5b6"}
Jul 02 19:09:09.957296 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:09.957262    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk" event={"ID":"8954e82e-6c64-4c92-814d-702dc9889237","Type":"ContainerStarted","Data":"6a990db2dfe427c1c70830ce23027894d5fbfac0bf216f8b93b215c3ab21fee3"}
Jul 02 19:09:10.962491 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:10.962446    2584 generic.go:356] "Generic (PLEG): container finished" podID="8954e82e-6c64-4c92-814d-702dc9889237" containerID="6a990db2dfe427c1c70830ce23027894d5fbfac0bf216f8b93b215c3ab21fee3" exitCode=0
Jul 02 19:09:10.962909 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:10.962537    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk" event={"ID":"8954e82e-6c64-4c92-814d-702dc9889237","Type":"ContainerDied","Data":"6a990db2dfe427c1c70830ce23027894d5fbfac0bf216f8b93b215c3ab21fee3"}
Jul 02 19:09:11.968110 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:11.968075    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk" event={"ID":"8954e82e-6c64-4c92-814d-702dc9889237","Type":"ContainerStarted","Data":"00e22162d4ef96641059318394dbc4a3aea93b43034a41d1f5c614898719f60a"}
Jul 02 19:09:11.968631 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:11.968224    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:22.973954 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:22.973926    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk"
Jul 02 19:09:22.998835 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:22.998774    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-cb89c69-4rjpk" podStartSLOduration=13.695151005 podStartE2EDuration="17.998754689s" totalImagesPullingTime="3.229496403s" totalInitContainerRuntime="1.074107281s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:11.99768378 +0000 UTC m=+331.852615551" watchObservedRunningTime="2026-07-02 19:09:22.998754689 +0000 UTC m=+342.853686459"
Jul 02 19:09:25.695501 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.695464    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8hgmc"]
Jul 02 19:09:25.704510 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.704492    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:25.707793 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.707772    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 02 19:09:25.707919 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.707780    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-f58sh\""
Jul 02 19:09:25.716687 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.716667    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8hgmc"]
Jul 02 19:09:25.833768 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.833728    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkzpp\" (UniqueName: \"kubernetes.io/projected/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-kube-api-access-tkzpp\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:25.833935 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.833775    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-cert\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:25.934325 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.934294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-cert\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:25.934506 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.934407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkzpp\" (UniqueName: \"kubernetes.io/projected/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-kube-api-access-tkzpp\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:25.934506 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:09:25.934456    2584 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 02 19:09:25.934619 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:09:25.934545    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-cert podName:cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323 nodeName:}" failed. No retries permitted until 2026-07-02 19:09:26.434520798 +0000 UTC m=+346.289452568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-cert") pod "odh-model-controller-7998c67dc4-8hgmc" (UID: "cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323") : secret "odh-model-controller-webhook-cert" not found
Jul 02 19:09:25.945681 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:25.945612    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkzpp\" (UniqueName: \"kubernetes.io/projected/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-kube-api-access-tkzpp\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:26.438776 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.438735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-cert\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:26.441243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.441213    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323-cert\") pod \"odh-model-controller-7998c67dc4-8hgmc\" (UID: \"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323\") " pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:26.556186 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.556155    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r4hsl"]
Jul 02 19:09:26.558726 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.558709    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.561272 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.561250    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qzzn2\""
Jul 02 19:09:26.561434 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.561418    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 02 19:09:26.577037 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.577006    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r4hsl"]
Jul 02 19:09:26.613796 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.613768    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:26.741806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.741768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h88hv\" (UniqueName: \"kubernetes.io/projected/9fda2112-24d7-42c5-a887-74e060e85ae4-kube-api-access-h88hv\") pod \"model-serving-api-54d89bd5fc-r4hsl\" (UID: \"9fda2112-24d7-42c5-a887-74e060e85ae4\") " pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.742231 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.741810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fda2112-24d7-42c5-a887-74e060e85ae4-tls-certs\") pod \"model-serving-api-54d89bd5fc-r4hsl\" (UID: \"9fda2112-24d7-42c5-a887-74e060e85ae4\") " pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.744490 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.744463    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8hgmc"]
Jul 02 19:09:26.746783 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:09:26.746756    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe96aa1_7ea1_4f2a_8d33_f3c3f6ed9323.slice/crio-d218c69421384f8cb17b92ee0319214a75be37314490583423b16228af06c859 WatchSource:0}: Error finding container d218c69421384f8cb17b92ee0319214a75be37314490583423b16228af06c859: Status 404 returned error can't find the container with id d218c69421384f8cb17b92ee0319214a75be37314490583423b16228af06c859
Jul 02 19:09:26.843085 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.843047    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h88hv\" (UniqueName: \"kubernetes.io/projected/9fda2112-24d7-42c5-a887-74e060e85ae4-kube-api-access-h88hv\") pod \"model-serving-api-54d89bd5fc-r4hsl\" (UID: \"9fda2112-24d7-42c5-a887-74e060e85ae4\") " pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.843085 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.843090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fda2112-24d7-42c5-a887-74e060e85ae4-tls-certs\") pod \"model-serving-api-54d89bd5fc-r4hsl\" (UID: \"9fda2112-24d7-42c5-a887-74e060e85ae4\") " pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.845468 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.845449    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fda2112-24d7-42c5-a887-74e060e85ae4-tls-certs\") pod \"model-serving-api-54d89bd5fc-r4hsl\" (UID: \"9fda2112-24d7-42c5-a887-74e060e85ae4\") " pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.852762 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.852742    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h88hv\" (UniqueName: \"kubernetes.io/projected/9fda2112-24d7-42c5-a887-74e060e85ae4-kube-api-access-h88hv\") pod \"model-serving-api-54d89bd5fc-r4hsl\" (UID: \"9fda2112-24d7-42c5-a887-74e060e85ae4\") " pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.868249 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.868222    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:26.993976 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:26.993941    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r4hsl"]
Jul 02 19:09:26.996401 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:09:26.996370    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fda2112_24d7_42c5_a887_74e060e85ae4.slice/crio-7ad41a69018806f11db4bbf5e8ba4478ff0dd1ccbc5071cca3475673cc176907 WatchSource:0}: Error finding container 7ad41a69018806f11db4bbf5e8ba4478ff0dd1ccbc5071cca3475673cc176907: Status 404 returned error can't find the container with id 7ad41a69018806f11db4bbf5e8ba4478ff0dd1ccbc5071cca3475673cc176907
Jul 02 19:09:27.021039 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:27.021008    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl" event={"ID":"9fda2112-24d7-42c5-a887-74e060e85ae4","Type":"ContainerStarted","Data":"7ad41a69018806f11db4bbf5e8ba4478ff0dd1ccbc5071cca3475673cc176907"}
Jul 02 19:09:27.022010 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:27.021987    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc" event={"ID":"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323","Type":"ContainerStarted","Data":"d218c69421384f8cb17b92ee0319214a75be37314490583423b16228af06c859"}
Jul 02 19:09:30.037493 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.037274    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl" event={"ID":"9fda2112-24d7-42c5-a887-74e060e85ae4","Type":"ContainerStarted","Data":"cd89a6c243e7db0fac4fa89d08debcb1ff8093bb20262a05bc344133e28741b6"}
Jul 02 19:09:30.037493 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.037396    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:30.057940 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.057897    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl" podStartSLOduration=2.006792691 podStartE2EDuration="4.057884504s" totalImagesPullingTime="2.051091813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:30.05705369 +0000 UTC m=+349.911985460" watchObservedRunningTime="2026-07-02 19:09:30.057884504 +0000 UTC m=+349.912816274"
Jul 02 19:09:30.770810 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.770772    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-578747f885-tgqz8"]
Jul 02 19:09:30.774216 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.774191    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.776629 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.776596    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 02 19:09:30.776742 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.776681    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 02 19:09:30.776843 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.776824    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 02 19:09:30.777655 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.777637    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 02 19:09:30.777655 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.777649    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-s2fdj\""
Jul 02 19:09:30.784728 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.784707    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-578747f885-tgqz8"]
Jul 02 19:09:30.877568 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.877536    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88504682-7871-4c59-95cb-cb412de7f35c-tmp\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.877736 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.877577    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl44j\" (UniqueName: \"kubernetes.io/projected/88504682-7871-4c59-95cb-cb412de7f35c-kube-api-access-hl44j\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.877736 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.877659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88504682-7871-4c59-95cb-cb412de7f35c-tls-certs\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.978969 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.978929    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88504682-7871-4c59-95cb-cb412de7f35c-tmp\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.979130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.978989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl44j\" (UniqueName: \"kubernetes.io/projected/88504682-7871-4c59-95cb-cb412de7f35c-kube-api-access-hl44j\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.979130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.979044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88504682-7871-4c59-95cb-cb412de7f35c-tls-certs\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.979211 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:09:30.979178    2584 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 02 19:09:30.979247 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:09:30.979243    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88504682-7871-4c59-95cb-cb412de7f35c-tls-certs podName:88504682-7871-4c59-95cb-cb412de7f35c nodeName:}" failed. No retries permitted until 2026-07-02 19:09:31.479222312 +0000 UTC m=+351.334154066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/88504682-7871-4c59-95cb-cb412de7f35c-tls-certs") pod "kube-auth-proxy-578747f885-tgqz8" (UID: "88504682-7871-4c59-95cb-cb412de7f35c") : secret "kube-auth-proxy-tls" not found
Jul 02 19:09:30.986172 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.986141    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88504682-7871-4c59-95cb-cb412de7f35c-tmp\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:30.990702 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:30.990669    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl44j\" (UniqueName: \"kubernetes.io/projected/88504682-7871-4c59-95cb-cb412de7f35c-kube-api-access-hl44j\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:31.042130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:31.042039    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc" event={"ID":"cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323","Type":"ContainerStarted","Data":"9307192b45705a095f55ce32183b2aa098cb1ffd8ce74a2fc182cd2084349a84"}
Jul 02 19:09:31.086850 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:31.086801    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc" podStartSLOduration=2.844900779 podStartE2EDuration="6.08678642s" totalImagesPullingTime="3.241885641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:31.084853351 +0000 UTC m=+350.939785133" watchObservedRunningTime="2026-07-02 19:09:31.08678642 +0000 UTC m=+350.941718190"
Jul 02 19:09:31.483066 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:31.483028    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88504682-7871-4c59-95cb-cb412de7f35c-tls-certs\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:31.485520 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:31.485496    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88504682-7871-4c59-95cb-cb412de7f35c-tls-certs\") pod \"kube-auth-proxy-578747f885-tgqz8\" (UID: \"88504682-7871-4c59-95cb-cb412de7f35c\") " pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:31.685067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:31.685029    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8"
Jul 02 19:09:31.823566 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:31.823511    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-578747f885-tgqz8"]
Jul 02 19:09:31.825575 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:09:31.825539    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88504682_7871_4c59_95cb_cb412de7f35c.slice/crio-d86c0f64f70b65b8a48a05ec132255525d302db80e1779d362c608268284c6c8 WatchSource:0}: Error finding container d86c0f64f70b65b8a48a05ec132255525d302db80e1779d362c608268284c6c8: Status 404 returned error can't find the container with id d86c0f64f70b65b8a48a05ec132255525d302db80e1779d362c608268284c6c8
Jul 02 19:09:32.046572 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.046487    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8" event={"ID":"88504682-7871-4c59-95cb-cb412de7f35c","Type":"ContainerStarted","Data":"d86c0f64f70b65b8a48a05ec132255525d302db80e1779d362c608268284c6c8"}
Jul 02 19:09:32.047013 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.046722    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:32.416703 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.416667    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"]
Jul 02 19:09:32.420829 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.420812    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.425522 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.425503    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 02 19:09:32.425913 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.425890    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cqbtw\""
Jul 02 19:09:32.425913 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.425906    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 02 19:09:32.426070 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.425929    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 02 19:09:32.429191 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.429168    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 02 19:09:32.440181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.440158    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"]
Jul 02 19:09:32.591907 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.591869    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.592086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.591914    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.592086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.591943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.592086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.592030    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a87ccab5-02ad-45f9-93b2-6681039a7b99-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.592202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.592087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.592202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.592137    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.592202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.592163    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24rq9\" (UniqueName: \"kubernetes.io/projected/a87ccab5-02ad-45f9-93b2-6681039a7b99-kube-api-access-24rq9\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692510    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692546    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24rq9\" (UniqueName: \"kubernetes.io/projected/a87ccab5-02ad-45f9-93b2-6681039a7b99-kube-api-access-24rq9\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692611    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692668    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a87ccab5-02ad-45f9-93b2-6681039a7b99-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.692978 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.692814    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.693427 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.693407    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.694996 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.694975    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.694996 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.694986    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.695184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.695167    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a87ccab5-02ad-45f9-93b2-6681039a7b99-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.695329 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.695308    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a87ccab5-02ad-45f9-93b2-6681039a7b99-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.701260 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.701235    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24rq9\" (UniqueName: \"kubernetes.io/projected/a87ccab5-02ad-45f9-93b2-6681039a7b99-kube-api-access-24rq9\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.701571 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.701551    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a87ccab5-02ad-45f9-93b2-6681039a7b99-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ks5b4\" (UID: \"a87ccab5-02ad-45f9-93b2-6681039a7b99\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.730445 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.730425    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:32.866226 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:32.866202    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"]
Jul 02 19:09:32.868625 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:09:32.868598    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87ccab5_02ad_45f9_93b2_6681039a7b99.slice/crio-eb854210ec9578e2fac4b6616f1bf3ddf95ac6adc64b3f3c2b0719bc79c6003a WatchSource:0}: Error finding container eb854210ec9578e2fac4b6616f1bf3ddf95ac6adc64b3f3c2b0719bc79c6003a: Status 404 returned error can't find the container with id eb854210ec9578e2fac4b6616f1bf3ddf95ac6adc64b3f3c2b0719bc79c6003a
Jul 02 19:09:33.053330 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:33.053233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4" event={"ID":"a87ccab5-02ad-45f9-93b2-6681039a7b99","Type":"ContainerStarted","Data":"eb854210ec9578e2fac4b6616f1bf3ddf95ac6adc64b3f3c2b0719bc79c6003a"}
Jul 02 19:09:36.161722 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:36.161679    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:09:36.161999 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:36.161766    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:09:37.073302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.073262    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4" event={"ID":"a87ccab5-02ad-45f9-93b2-6681039a7b99","Type":"ContainerStarted","Data":"272658dc83e38fd85c2be0c35d96754fe5b6ae965ff4eff1ed3a5df18ca08634"}
Jul 02 19:09:37.073515 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.073489    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:37.074871 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.074847    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8" event={"ID":"88504682-7871-4c59-95cb-cb412de7f35c","Type":"ContainerStarted","Data":"e45d50c8a8a15a72d62d2ca81ded80ecc9794d734780e6365777682b066c6a91"}
Jul 02 19:09:37.075242 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.075216    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-ks5b4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 02 19:09:37.075308 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.075268    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4" podUID="a87ccab5-02ad-45f9-93b2-6681039a7b99" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 19:09:37.128772 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.128723    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4" podStartSLOduration=1.838498914 podStartE2EDuration="5.128707659s" totalImagesPullingTime="3.290208745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:37.126848448 +0000 UTC m=+356.981780218" watchObservedRunningTime="2026-07-02 19:09:37.128707659 +0000 UTC m=+356.983639430"
Jul 02 19:09:37.153777 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:37.153737    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-578747f885-tgqz8" podStartSLOduration=2.817222979 podStartE2EDuration="7.153723495s" totalImagesPullingTime="4.336500516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:37.152550924 +0000 UTC m=+357.007482692" watchObservedRunningTime="2026-07-02 19:09:37.153723495 +0000 UTC m=+357.008655265"
Jul 02 19:09:38.078806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.078777    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ks5b4"
Jul 02 19:09:38.165056 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.165024    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"]
Jul 02 19:09:38.168284 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.168266    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.172245 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.172225    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zh6pz\""
Jul 02 19:09:38.197554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.197526    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"]
Jul 02 19:09:38.231707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231678    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.231865 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231713    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.231865 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.231865 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.231865 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231797    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh652\" (UniqueName: \"kubernetes.io/projected/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-kube-api-access-zh652\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.232067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231868    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.232067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231920    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.232067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231946    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.232067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.231980    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332469    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332714 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332525    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332714 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332714 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332877 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332877 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332888    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh652\" (UniqueName: \"kubernetes.io/projected/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-kube-api-access-zh652\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332923    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.332975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332953    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.333127 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.332974    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.333177 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.333158    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.333589 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.333551    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.333662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.333624    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.333662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.333649    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.334991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.334965    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.335768 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.335749    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.356666 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.356643    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh652\" (UniqueName: \"kubernetes.io/projected/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-kube-api-access-zh652\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.370872 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.370847    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cedd2fc4-ac14-437d-94dd-61a3fc625a1a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb67864b99\" (UID: \"cedd2fc4-ac14-437d-94dd-61a3fc625a1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.478906 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.478866    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:38.636375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:38.636332    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"]
Jul 02 19:09:38.637984 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:09:38.637956    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedd2fc4_ac14_437d_94dd_61a3fc625a1a.slice/crio-5013be88ac7f44a79909b27404bf569d4406df9280bf7d9a6560eeeb619c4961 WatchSource:0}: Error finding container 5013be88ac7f44a79909b27404bf569d4406df9280bf7d9a6560eeeb619c4961: Status 404 returned error can't find the container with id 5013be88ac7f44a79909b27404bf569d4406df9280bf7d9a6560eeeb619c4961
Jul 02 19:09:39.084297 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:39.084261    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99" event={"ID":"cedd2fc4-ac14-437d-94dd-61a3fc625a1a","Type":"ContainerStarted","Data":"5013be88ac7f44a79909b27404bf569d4406df9280bf7d9a6560eeeb619c4961"}
Jul 02 19:09:41.047077 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:41.047042    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-r4hsl"
Jul 02 19:09:41.874813 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:41.874778    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:09:41.874896 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:41.874851    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:09:41.874896 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:41.874882    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:09:42.100831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:42.100789    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99" event={"ID":"cedd2fc4-ac14-437d-94dd-61a3fc625a1a","Type":"ContainerStarted","Data":"d729e1aa5ebbb9c1ed4717f7f1f4058cab578af7889559a3f7016a704f650dc7"}
Jul 02 19:09:42.147789 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:42.147660    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99" podStartSLOduration=0.913217859 podStartE2EDuration="4.147640427s" totalImagesPullingTime="3.234422568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:42.146081431 +0000 UTC m=+362.001013201" watchObservedRunningTime="2026-07-02 19:09:42.147640427 +0000 UTC m=+362.002572197"
Jul 02 19:09:42.479942 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:42.479853    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:42.484647 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:42.484623    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:43.055684 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:43.055656    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8hgmc"
Jul 02 19:09:43.105562 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:43.105529    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:09:43.106635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:09:43.106618    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb67864b99"
Jul 02 19:11:10.074426 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.074390    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"]
Jul 02 19:11:10.076531 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.076514    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.079061 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.079039    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mb6l4\""
Jul 02 19:11:10.079452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.079433    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 02 19:11:10.079562 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.079442    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 02 19:11:10.091625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.091598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"]
Jul 02 19:11:10.194584 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.194547    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3031bc78-ce8a-4d16-9e4f-651405427f1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.194766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.194601    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cq52\" (UniqueName: \"kubernetes.io/projected/3031bc78-ce8a-4d16-9e4f-651405427f1f-kube-api-access-2cq52\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.295404 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.295364    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3031bc78-ce8a-4d16-9e4f-651405427f1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.295565 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.295435    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cq52\" (UniqueName: \"kubernetes.io/projected/3031bc78-ce8a-4d16-9e4f-651405427f1f-kube-api-access-2cq52\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.295720 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.295703    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3031bc78-ce8a-4d16-9e4f-651405427f1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.306673 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.306647    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cq52\" (UniqueName: \"kubernetes.io/projected/3031bc78-ce8a-4d16-9e4f-651405427f1f-kube-api-access-2cq52\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.386661 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.386627    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:10.522895 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:10.522861    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"]
Jul 02 19:11:10.528001 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:11:10.527967    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3031bc78_ce8a_4d16_9e4f_651405427f1f.slice/crio-e6d4fed8b50e4260d848c30c6531c13ae302ee6d0519729f2460d75cc89393db WatchSource:0}: Error finding container e6d4fed8b50e4260d848c30c6531c13ae302ee6d0519729f2460d75cc89393db: Status 404 returned error can't find the container with id e6d4fed8b50e4260d848c30c6531c13ae302ee6d0519729f2460d75cc89393db
Jul 02 19:11:11.457110 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:11.457077    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v" event={"ID":"3031bc78-ce8a-4d16-9e4f-651405427f1f","Type":"ContainerStarted","Data":"e6d4fed8b50e4260d848c30c6531c13ae302ee6d0519729f2460d75cc89393db"}
Jul 02 19:11:17.487117 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:17.487075    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v" event={"ID":"3031bc78-ce8a-4d16-9e4f-651405427f1f","Type":"ContainerStarted","Data":"9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d"}
Jul 02 19:11:17.487571 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:17.487167    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:17.522313 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:17.522266    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v" podStartSLOduration=1.480815486 podStartE2EDuration="7.522251813s" totalImagesPullingTime="6.041436327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:17.521187794 +0000 UTC m=+457.376119563" watchObservedRunningTime="2026-07-02 19:11:17.522251813 +0000 UTC m=+457.377183582"
Jul 02 19:11:28.493758 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:28.493724    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.221335 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.221294    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"]
Jul 02 19:11:30.223653 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.223634    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.240822 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.240797    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"]
Jul 02 19:11:30.265709 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.265679    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-965vn\" (UniqueName: \"kubernetes.io/projected/90a6eac9-7cce-494e-b5c1-633bb2909335-kube-api-access-965vn\") pod \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.265851 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.265804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a6eac9-7cce-494e-b5c1-633bb2909335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.360092 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.360061    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"]
Jul 02 19:11:30.360318 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.360299    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-965vn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335"
Jul 02 19:11:30.366835 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.366807    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a6eac9-7cce-494e-b5c1-633bb2909335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.366920 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.366845    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-965vn\" (UniqueName: \"kubernetes.io/projected/90a6eac9-7cce-494e-b5c1-633bb2909335-kube-api-access-965vn\") pod \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.367219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.367195    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a6eac9-7cce-494e-b5c1-633bb2909335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.378409 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.378386    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-965vn\" (UniqueName: \"kubernetes.io/projected/90a6eac9-7cce-494e-b5c1-633bb2909335-kube-api-access-965vn\") pod \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.453211 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.453179    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"]
Jul 02 19:11:30.463803 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.463774    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"]
Jul 02 19:11:30.478176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.478115    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"]
Jul 02 19:11:30.478312 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.478286    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" containerName="manager" containerID="cri-o://9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d" gracePeriod=2
Jul 02 19:11:30.491611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.491589    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"]
Jul 02 19:11:30.493639 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.493545    2584 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3031bc78-ce8a-4d16-9e4f-651405427f1f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\": pods \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.507138 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.507107    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"]
Jul 02 19:11:30.507819 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.507801    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" containerName="manager"
Jul 02 19:11:30.507877 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.507823    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" containerName="manager"
Jul 02 19:11:30.507985 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.507972    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" containerName="manager"
Jul 02 19:11:30.510475 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.510454    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.513322 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.513152    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.535536 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.535511    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"]
Jul 02 19:11:30.543292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.543267    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.568373 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.568326    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.568591 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.568565    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75b8af3-014f-43dd-ad71-3b53bf73c280-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tchtc\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.568708 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.568691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxnfn\" (UniqueName: \"kubernetes.io/projected/f75b8af3-014f-43dd-ad71-3b53bf73c280-kube-api-access-bxnfn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tchtc\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.572492 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.572465    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.578659 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.578641    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.581702 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.581674    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.588651 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.588624    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.669592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.669565    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-965vn\" (UniqueName: \"kubernetes.io/projected/90a6eac9-7cce-494e-b5c1-633bb2909335-kube-api-access-965vn\") pod \"90a6eac9-7cce-494e-b5c1-633bb2909335\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") "
Jul 02 19:11:30.669724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.669625    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a6eac9-7cce-494e-b5c1-633bb2909335-extensions-socket-volume\") pod \"90a6eac9-7cce-494e-b5c1-633bb2909335\" (UID: \"90a6eac9-7cce-494e-b5c1-633bb2909335\") "
Jul 02 19:11:30.669786 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.669732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxnfn\" (UniqueName: \"kubernetes.io/projected/f75b8af3-014f-43dd-ad71-3b53bf73c280-kube-api-access-bxnfn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tchtc\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.669842 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.669785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75b8af3-014f-43dd-ad71-3b53bf73c280-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tchtc\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.669961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.669927    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90a6eac9-7cce-494e-b5c1-633bb2909335-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "90a6eac9-7cce-494e-b5c1-633bb2909335" (UID: "90a6eac9-7cce-494e-b5c1-633bb2909335"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:11:30.670179 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.670161    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75b8af3-014f-43dd-ad71-3b53bf73c280-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tchtc\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.671649 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.671620    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90a6eac9-7cce-494e-b5c1-633bb2909335-kube-api-access-965vn" (OuterVolumeSpecName: "kube-api-access-965vn") pod "90a6eac9-7cce-494e-b5c1-633bb2909335" (UID: "90a6eac9-7cce-494e-b5c1-633bb2909335"). InnerVolumeSpecName "kube-api-access-965vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:11:30.682767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.682745    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxnfn\" (UniqueName: \"kubernetes.io/projected/f75b8af3-014f-43dd-ad71-3b53bf73c280-kube-api-access-bxnfn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tchtc\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:30.716644 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.716623    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.723288 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.723264    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.731891 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.731847    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.770294 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.770274    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3031bc78-ce8a-4d16-9e4f-651405427f1f-extensions-socket-volume\") pod \"3031bc78-ce8a-4d16-9e4f-651405427f1f\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") "
Jul 02 19:11:30.770442 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.770385    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cq52\" (UniqueName: \"kubernetes.io/projected/3031bc78-ce8a-4d16-9e4f-651405427f1f-kube-api-access-2cq52\") pod \"3031bc78-ce8a-4d16-9e4f-651405427f1f\" (UID: \"3031bc78-ce8a-4d16-9e4f-651405427f1f\") "
Jul 02 19:11:30.770597 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.770573    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-965vn\" (UniqueName: \"kubernetes.io/projected/90a6eac9-7cce-494e-b5c1-633bb2909335-kube-api-access-965vn\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:11:30.770649 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.770604    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a6eac9-7cce-494e-b5c1-633bb2909335-extensions-socket-volume\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:11:30.770730 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.770700    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3031bc78-ce8a-4d16-9e4f-651405427f1f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3031bc78-ce8a-4d16-9e4f-651405427f1f" (UID: "3031bc78-ce8a-4d16-9e4f-651405427f1f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:11:30.772575 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.772550    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3031bc78-ce8a-4d16-9e4f-651405427f1f-kube-api-access-2cq52" (OuterVolumeSpecName: "kube-api-access-2cq52") pod "3031bc78-ce8a-4d16-9e4f-651405427f1f" (UID: "3031bc78-ce8a-4d16-9e4f-651405427f1f"). InnerVolumeSpecName "kube-api-access-2cq52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:11:30.774871 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.774841    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:30.775878 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.775860    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" path="/var/lib/kubelet/pods/3031bc78-ce8a-4d16-9e4f-651405427f1f/volumes"
Jul 02 19:11:30.776184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.776172    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335" path="/var/lib/kubelet/pods/90a6eac9-7cce-494e-b5c1-633bb2909335/volumes"
Jul 02 19:11:30.777087 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:30.777063    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sjm2v\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="3031bc78-ce8a-4d16-9e4f-651405427f1f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:30.871389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.871330    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cq52\" (UniqueName: \"kubernetes.io/projected/3031bc78-ce8a-4d16-9e4f-651405427f1f-kube-api-access-2cq52\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:11:30.871389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.871379    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3031bc78-ce8a-4d16-9e4f-651405427f1f-extensions-socket-volume\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:11:30.876293 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:30.876265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:31.017409 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.017384    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"]
Jul 02 19:11:31.019519 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:11:31.019494    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75b8af3_014f_43dd_ad71_3b53bf73c280.slice/crio-a253ef9abeeee99947f88ff93582942469fda331aca0b82857735228accb7498 WatchSource:0}: Error finding container a253ef9abeeee99947f88ff93582942469fda331aca0b82857735228accb7498: Status 404 returned error can't find the container with id a253ef9abeeee99947f88ff93582942469fda331aca0b82857735228accb7498
Jul 02 19:11:31.548281 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.548244    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc" event={"ID":"f75b8af3-014f-43dd-ad71-3b53bf73c280","Type":"ContainerStarted","Data":"52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26"}
Jul 02 19:11:31.548727 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.548301    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc" event={"ID":"f75b8af3-014f-43dd-ad71-3b53bf73c280","Type":"ContainerStarted","Data":"a253ef9abeeee99947f88ff93582942469fda331aca0b82857735228accb7498"}
Jul 02 19:11:31.548727 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.548331    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:31.549616 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.549593    2584 generic.go:356] "Generic (PLEG): container finished" podID="3031bc78-ce8a-4d16-9e4f-651405427f1f" containerID="9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d" exitCode=0
Jul 02 19:11:31.549707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.549630    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sjm2v"
Jul 02 19:11:31.549707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.549682    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:31.549707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.549689    2584 scope.go:117] "RemoveContainer" containerID="9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d"
Jul 02 19:11:31.558885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.558865    2584 scope.go:117] "RemoveContainer" containerID="9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d"
Jul 02 19:11:31.559147 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:31.559129    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d\": container with ID starting with 9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d not found: ID does not exist" containerID="9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d"
Jul 02 19:11:31.559196 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:31.559155    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d"} err="failed to get container status \"9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d\": rpc error: code = NotFound desc = could not find container \"9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d\": container with ID starting with 9a18c9be64dca70ab2b2139e8ce7fe271711b883f442f3ba694749c747d8544d not found: ID does not exist"
Jul 02 19:11:31.612530 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:11:31.612494    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-7ddh6\" is forbidden: User \"system:node:ip-10-0-135-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-120.ec2.internal' and this object" podUID="90a6eac9-7cce-494e-b5c1-633bb2909335" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7ddh6"
Jul 02 19:11:42.557119 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:42.557087    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:11:42.580021 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:42.579969    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc" podStartSLOduration=12.579941066 podStartE2EDuration="12.579941066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:31.602310149 +0000 UTC m=+471.457241936" watchObservedRunningTime="2026-07-02 19:11:42.579941066 +0000 UTC m=+482.434872835"
Jul 02 19:11:59.710419 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:59.710375    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"]
Jul 02 19:11:59.710941 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:59.710730    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc" podUID="f75b8af3-014f-43dd-ad71-3b53bf73c280" containerName="manager" containerID="cri-o://52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26" gracePeriod=10
Jul 02 19:11:59.952872 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:11:59.952847    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:12:00.112095 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.112055    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75b8af3-014f-43dd-ad71-3b53bf73c280-extensions-socket-volume\") pod \"f75b8af3-014f-43dd-ad71-3b53bf73c280\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") "
Jul 02 19:12:00.112262 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.112125    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxnfn\" (UniqueName: \"kubernetes.io/projected/f75b8af3-014f-43dd-ad71-3b53bf73c280-kube-api-access-bxnfn\") pod \"f75b8af3-014f-43dd-ad71-3b53bf73c280\" (UID: \"f75b8af3-014f-43dd-ad71-3b53bf73c280\") "
Jul 02 19:12:00.112438 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.112412    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75b8af3-014f-43dd-ad71-3b53bf73c280-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f75b8af3-014f-43dd-ad71-3b53bf73c280" (UID: "f75b8af3-014f-43dd-ad71-3b53bf73c280"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:12:00.114145 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.114123    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f75b8af3-014f-43dd-ad71-3b53bf73c280-kube-api-access-bxnfn" (OuterVolumeSpecName: "kube-api-access-bxnfn") pod "f75b8af3-014f-43dd-ad71-3b53bf73c280" (UID: "f75b8af3-014f-43dd-ad71-3b53bf73c280"). InnerVolumeSpecName "kube-api-access-bxnfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:12:00.213316 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.213266    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f75b8af3-014f-43dd-ad71-3b53bf73c280-extensions-socket-volume\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:00.213316 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.213310    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxnfn\" (UniqueName: \"kubernetes.io/projected/f75b8af3-014f-43dd-ad71-3b53bf73c280-kube-api-access-bxnfn\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:00.674317 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.674281    2584 generic.go:356] "Generic (PLEG): container finished" podID="f75b8af3-014f-43dd-ad71-3b53bf73c280" containerID="52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26" exitCode=0
Jul 02 19:12:00.674498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.674321    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"
Jul 02 19:12:00.674498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.674372    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc" event={"ID":"f75b8af3-014f-43dd-ad71-3b53bf73c280","Type":"ContainerDied","Data":"52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26"}
Jul 02 19:12:00.674498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.674420    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc" event={"ID":"f75b8af3-014f-43dd-ad71-3b53bf73c280","Type":"ContainerDied","Data":"a253ef9abeeee99947f88ff93582942469fda331aca0b82857735228accb7498"}
Jul 02 19:12:00.674498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.674440    2584 scope.go:117] "RemoveContainer" containerID="52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26"
Jul 02 19:12:00.683216 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.683199    2584 scope.go:117] "RemoveContainer" containerID="52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26"
Jul 02 19:12:00.683485 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:12:00.683465    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26\": container with ID starting with 52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26 not found: ID does not exist" containerID="52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26"
Jul 02 19:12:00.683537 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.683493    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26"} err="failed to get container status \"52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26\": rpc error: code = NotFound desc = could not find container \"52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26\": container with ID starting with 52ef659395467820af0fc7ca30bd9562815eeb9c2c7ebc903543fb7c7b089b26 not found: ID does not exist"
Jul 02 19:12:00.704727 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.704706    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"]
Jul 02 19:12:00.708181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.708161    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tchtc"]
Jul 02 19:12:00.776110 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:00.776078    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f75b8af3-014f-43dd-ad71-3b53bf73c280" path="/var/lib/kubelet/pods/f75b8af3-014f-43dd-ad71-3b53bf73c280/volumes"
Jul 02 19:12:16.130041 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.130008    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"]
Jul 02 19:12:16.130554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.130284    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f75b8af3-014f-43dd-ad71-3b53bf73c280" containerName="manager"
Jul 02 19:12:16.130554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.130296    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f75b8af3-014f-43dd-ad71-3b53bf73c280" containerName="manager"
Jul 02 19:12:16.130554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.130392    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f75b8af3-014f-43dd-ad71-3b53bf73c280" containerName="manager"
Jul 02 19:12:16.132520 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.132504    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.135140 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.135119    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z8rsd\""
Jul 02 19:12:16.141700 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141680    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.141808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141716    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.141808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141756    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.141808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141794    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.141918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141858    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.141918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.141918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mv8j\" (UniqueName: \"kubernetes.io/projected/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-kube-api-access-2mv8j\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.142021 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.141940    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.142021 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.142000    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.151244 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.151220    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"]
Jul 02 19:12:16.243389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243349    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243417    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243458    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243518    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mv8j\" (UniqueName: \"kubernetes.io/projected/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-kube-api-access-2mv8j\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243775 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243564    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243775 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243775 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243775 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243814    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.243975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.243873    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.244121 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.244095    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.244190 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.244134    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.244245 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.244225    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.245915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.245892    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.245915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.245903    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.252202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.252175    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.252415 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.252398    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mv8j\" (UniqueName: \"kubernetes.io/projected/da9439cf-a90e-4a70-9b66-5ccb6f5feb9e-kube-api-access-2mv8j\") pod \"maas-default-gateway-openshift-default-5896d4dd49-5h8rj\" (UID: \"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.441883 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.441799    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:16.576024 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.576000    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"]
Jul 02 19:12:16.577850 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:12:16.577822    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9439cf_a90e_4a70_9b66_5ccb6f5feb9e.slice/crio-02037ca42331624048c04666b955fe32190662f472d87f77c8d63ea7083688d5 WatchSource:0}: Error finding container 02037ca42331624048c04666b955fe32190662f472d87f77c8d63ea7083688d5: Status 404 returned error can't find the container with id 02037ca42331624048c04666b955fe32190662f472d87f77c8d63ea7083688d5
Jul 02 19:12:16.579834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.579801    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:12:16.579926 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.579890    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:12:16.579986 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.579947    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:12:16.738821 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.738729    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" event={"ID":"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e","Type":"ContainerStarted","Data":"c088f853c837ea3d56a594fbd1a5ca276e4980dcb64e87cb8237c2a965b4f847"}
Jul 02 19:12:16.738821 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.738786    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" event={"ID":"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e","Type":"ContainerStarted","Data":"02037ca42331624048c04666b955fe32190662f472d87f77c8d63ea7083688d5"}
Jul 02 19:12:16.764744 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:16.764687    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" podStartSLOduration=0.764667514 podStartE2EDuration="764.667514ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:16.763988901 +0000 UTC m=+516.618920672" watchObservedRunningTime="2026-07-02 19:12:16.764667514 +0000 UTC m=+516.619599286"
Jul 02 19:12:17.442584 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.442547    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:17.447192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.447167    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:17.743115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.743015    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:17.744049 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.744030    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:12:17.933874 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.933837    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:17.936135 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.936119    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:17.939505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.939474    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bfbmt\""
Jul 02 19:12:17.939782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.939759    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 02 19:12:17.939882 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.939759    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 02 19:12:17.940370 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.940331    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 02 19:12:17.956289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.956264    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcg6f\" (UniqueName: \"kubernetes.io/projected/d4b6daac-195d-412c-b28b-1d88c618ed72-kube-api-access-jcg6f\") pod \"limitador-limitador-64c94fd64f-nwbxl\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:17.956424 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.956323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4b6daac-195d-412c-b28b-1d88c618ed72-config-file\") pod \"limitador-limitador-64c94fd64f-nwbxl\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:17.969886 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:17.969851    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:18.055820 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.055741    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:18.057575 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.057553    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcg6f\" (UniqueName: \"kubernetes.io/projected/d4b6daac-195d-412c-b28b-1d88c618ed72-kube-api-access-jcg6f\") pod \"limitador-limitador-64c94fd64f-nwbxl\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:18.057644 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.057603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4b6daac-195d-412c-b28b-1d88c618ed72-config-file\") pod \"limitador-limitador-64c94fd64f-nwbxl\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:18.058226 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.058205    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4b6daac-195d-412c-b28b-1d88c618ed72-config-file\") pod \"limitador-limitador-64c94fd64f-nwbxl\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:18.080589 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.080561    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcg6f\" (UniqueName: \"kubernetes.io/projected/d4b6daac-195d-412c-b28b-1d88c618ed72-kube-api-access-jcg6f\") pod \"limitador-limitador-64c94fd64f-nwbxl\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:18.247958 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.247918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:18.390502 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.390464    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:18.393450 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:12:18.393425    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b6daac_195d_412c_b28b_1d88c618ed72.slice/crio-493f28607371d2fccd65a19b13749141e55cc251a3e9304ff8db35b6fdbc8d75 WatchSource:0}: Error finding container 493f28607371d2fccd65a19b13749141e55cc251a3e9304ff8db35b6fdbc8d75: Status 404 returned error can't find the container with id 493f28607371d2fccd65a19b13749141e55cc251a3e9304ff8db35b6fdbc8d75
Jul 02 19:12:18.748248 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.748215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl" event={"ID":"d4b6daac-195d-412c-b28b-1d88c618ed72","Type":"ContainerStarted","Data":"493f28607371d2fccd65a19b13749141e55cc251a3e9304ff8db35b6fdbc8d75"}
Jul 02 19:12:18.778922 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.778897    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7kjbb"]
Jul 02 19:12:18.782176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.782156    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:18.784825 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.784806    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m5cfg\""
Jul 02 19:12:18.788659 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.788638    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7kjbb"]
Jul 02 19:12:18.864521 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.864483    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24kj\" (UniqueName: \"kubernetes.io/projected/d2c21b75-c3be-40c6-b90c-2aa64877adf4-kube-api-access-v24kj\") pod \"authorino-58cd955675-7kjbb\" (UID: \"d2c21b75-c3be-40c6-b90c-2aa64877adf4\") " pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:18.964991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.964958    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24kj\" (UniqueName: \"kubernetes.io/projected/d2c21b75-c3be-40c6-b90c-2aa64877adf4-kube-api-access-v24kj\") pod \"authorino-58cd955675-7kjbb\" (UID: \"d2c21b75-c3be-40c6-b90c-2aa64877adf4\") " pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:18.988626 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:18.988594    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24kj\" (UniqueName: \"kubernetes.io/projected/d2c21b75-c3be-40c6-b90c-2aa64877adf4-kube-api-access-v24kj\") pod \"authorino-58cd955675-7kjbb\" (UID: \"d2c21b75-c3be-40c6-b90c-2aa64877adf4\") " pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:19.092462 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:19.092433    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:19.316283 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:19.316190    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7kjbb"]
Jul 02 19:12:19.754532 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:19.754497    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7kjbb" event={"ID":"d2c21b75-c3be-40c6-b90c-2aa64877adf4","Type":"ContainerStarted","Data":"b221351d5729bb0dcdea9d74bc76137147a4554900bd4f33df90f613880abe60"}
Jul 02 19:12:23.689876 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:23.689841    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7kjbb"]
Jul 02 19:12:23.778660 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:23.778612    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7kjbb" event={"ID":"d2c21b75-c3be-40c6-b90c-2aa64877adf4","Type":"ContainerStarted","Data":"e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c"}
Jul 02 19:12:23.780263 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:23.780236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl" event={"ID":"d4b6daac-195d-412c-b28b-1d88c618ed72","Type":"ContainerStarted","Data":"d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed"}
Jul 02 19:12:23.780412 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:23.780325    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:23.794798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:23.794744    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7kjbb" podStartSLOduration=2.058614231 podStartE2EDuration="5.79472503s" totalImagesPullingTime="3.736110799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:23.794624783 +0000 UTC m=+523.649556555" watchObservedRunningTime="2026-07-02 19:12:23.79472503 +0000 UTC m=+523.649656801"
Jul 02 19:12:23.812531 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:23.812485    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl" podStartSLOduration=2.197187578 podStartE2EDuration="6.812467075s" totalImagesPullingTime="4.615279497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:23.811496257 +0000 UTC m=+523.666428028" watchObservedRunningTime="2026-07-02 19:12:23.812467075 +0000 UTC m=+523.667398845"
Jul 02 19:12:24.784387 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:24.784313    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7kjbb" podUID="d2c21b75-c3be-40c6-b90c-2aa64877adf4" containerName="authorino" containerID="cri-o://e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c" gracePeriod=30
Jul 02 19:12:25.022218 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.022193    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:25.118124 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.118091    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v24kj\" (UniqueName: \"kubernetes.io/projected/d2c21b75-c3be-40c6-b90c-2aa64877adf4-kube-api-access-v24kj\") pod \"d2c21b75-c3be-40c6-b90c-2aa64877adf4\" (UID: \"d2c21b75-c3be-40c6-b90c-2aa64877adf4\") "
Jul 02 19:12:25.120134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.120106    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c21b75-c3be-40c6-b90c-2aa64877adf4-kube-api-access-v24kj" (OuterVolumeSpecName: "kube-api-access-v24kj") pod "d2c21b75-c3be-40c6-b90c-2aa64877adf4" (UID: "d2c21b75-c3be-40c6-b90c-2aa64877adf4"). InnerVolumeSpecName "kube-api-access-v24kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:12:25.218880 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.218844    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v24kj\" (UniqueName: \"kubernetes.io/projected/d2c21b75-c3be-40c6-b90c-2aa64877adf4-kube-api-access-v24kj\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:25.789104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.789064    2584 generic.go:356] "Generic (PLEG): container finished" podID="d2c21b75-c3be-40c6-b90c-2aa64877adf4" containerID="e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c" exitCode=0
Jul 02 19:12:25.789555 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.789114    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7kjbb"
Jul 02 19:12:25.789555 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.789141    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7kjbb" event={"ID":"d2c21b75-c3be-40c6-b90c-2aa64877adf4","Type":"ContainerDied","Data":"e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c"}
Jul 02 19:12:25.789555 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.789175    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7kjbb" event={"ID":"d2c21b75-c3be-40c6-b90c-2aa64877adf4","Type":"ContainerDied","Data":"b221351d5729bb0dcdea9d74bc76137147a4554900bd4f33df90f613880abe60"}
Jul 02 19:12:25.789555 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.789197    2584 scope.go:117] "RemoveContainer" containerID="e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c"
Jul 02 19:12:25.797729 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.797710    2584 scope.go:117] "RemoveContainer" containerID="e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c"
Jul 02 19:12:25.798014 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:12:25.797981    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c\": container with ID starting with e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c not found: ID does not exist" containerID="e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c"
Jul 02 19:12:25.798063 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.798022    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c"} err="failed to get container status \"e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c\": rpc error: code = NotFound desc = could not find container \"e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c\": container with ID starting with e48e749c32c01959d2fbb0e1ca4cf6b8edee9fcec902234d1d2bb2ff9eafef5c not found: ID does not exist"
Jul 02 19:12:25.812164 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.812136    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7kjbb"]
Jul 02 19:12:25.816503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:25.816481    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7kjbb"]
Jul 02 19:12:26.777050 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:26.777015    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2c21b75-c3be-40c6-b90c-2aa64877adf4" path="/var/lib/kubelet/pods/d2c21b75-c3be-40c6-b90c-2aa64877adf4/volumes"
Jul 02 19:12:34.713465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:34.713383    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:34.713860 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:34.713627    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl" podUID="d4b6daac-195d-412c-b28b-1d88c618ed72" containerName="limitador" containerID="cri-o://d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed" gracePeriod=30
Jul 02 19:12:34.714305 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:34.714286    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:35.660194 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.660169    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:35.704137 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.704104    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4b6daac-195d-412c-b28b-1d88c618ed72-config-file\") pod \"d4b6daac-195d-412c-b28b-1d88c618ed72\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") "
Jul 02 19:12:35.704297 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.704158    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcg6f\" (UniqueName: \"kubernetes.io/projected/d4b6daac-195d-412c-b28b-1d88c618ed72-kube-api-access-jcg6f\") pod \"d4b6daac-195d-412c-b28b-1d88c618ed72\" (UID: \"d4b6daac-195d-412c-b28b-1d88c618ed72\") "
Jul 02 19:12:35.704539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.704515    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b6daac-195d-412c-b28b-1d88c618ed72-config-file" (OuterVolumeSpecName: "config-file") pod "d4b6daac-195d-412c-b28b-1d88c618ed72" (UID: "d4b6daac-195d-412c-b28b-1d88c618ed72"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:12:35.706119 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.706104    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b6daac-195d-412c-b28b-1d88c618ed72-kube-api-access-jcg6f" (OuterVolumeSpecName: "kube-api-access-jcg6f") pod "d4b6daac-195d-412c-b28b-1d88c618ed72" (UID: "d4b6daac-195d-412c-b28b-1d88c618ed72"). InnerVolumeSpecName "kube-api-access-jcg6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:12:35.805566 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.805483    2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4b6daac-195d-412c-b28b-1d88c618ed72-config-file\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:35.805566 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.805512    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcg6f\" (UniqueName: \"kubernetes.io/projected/d4b6daac-195d-412c-b28b-1d88c618ed72-kube-api-access-jcg6f\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:35.832869 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.832834    2584 generic.go:356] "Generic (PLEG): container finished" podID="d4b6daac-195d-412c-b28b-1d88c618ed72" containerID="d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed" exitCode=0
Jul 02 19:12:35.833004 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.832882    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"
Jul 02 19:12:35.833004 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.832927    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl" event={"ID":"d4b6daac-195d-412c-b28b-1d88c618ed72","Type":"ContainerDied","Data":"d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed"}
Jul 02 19:12:35.833004 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.832974    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nwbxl" event={"ID":"d4b6daac-195d-412c-b28b-1d88c618ed72","Type":"ContainerDied","Data":"493f28607371d2fccd65a19b13749141e55cc251a3e9304ff8db35b6fdbc8d75"}
Jul 02 19:12:35.833004 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.832996    2584 scope.go:117] "RemoveContainer" containerID="d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed"
Jul 02 19:12:35.841771 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.841751    2584 scope.go:117] "RemoveContainer" containerID="d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed"
Jul 02 19:12:35.842024 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:12:35.842002    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed\": container with ID starting with d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed not found: ID does not exist" containerID="d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed"
Jul 02 19:12:35.842083 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.842031    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed"} err="failed to get container status \"d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed\": rpc error: code = NotFound desc = could not find container \"d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed\": container with ID starting with d1758169ae57acd363459a50184c26d930719a5e77a2a1f5884f234fe03b23ed not found: ID does not exist"
Jul 02 19:12:35.854552 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.854523    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:35.860783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:35.860759    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nwbxl"]
Jul 02 19:12:36.776267 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:36.776231    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4b6daac-195d-412c-b28b-1d88c618ed72" path="/var/lib/kubelet/pods/d4b6daac-195d-412c-b28b-1d88c618ed72/volumes"
Jul 02 19:12:39.493599 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.493561    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"]
Jul 02 19:12:39.494033 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.494017    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c21b75-c3be-40c6-b90c-2aa64877adf4" containerName="authorino"
Jul 02 19:12:39.494086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.494037    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c21b75-c3be-40c6-b90c-2aa64877adf4" containerName="authorino"
Jul 02 19:12:39.494086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.494063    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b6daac-195d-412c-b28b-1d88c618ed72" containerName="limitador"
Jul 02 19:12:39.494086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.494071    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b6daac-195d-412c-b28b-1d88c618ed72" containerName="limitador"
Jul 02 19:12:39.494180 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.494138    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c21b75-c3be-40c6-b90c-2aa64877adf4" containerName="authorino"
Jul 02 19:12:39.494180 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.494151    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b6daac-195d-412c-b28b-1d88c618ed72" containerName="limitador"
Jul 02 19:12:39.498802 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.498771    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.500638 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.500613    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"]
Jul 02 19:12:39.501409 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.501387    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 02 19:12:39.501409 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.501400    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 02 19:12:39.502067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.502049    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6q4r6\""
Jul 02 19:12:39.502067 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.502060    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 02 19:12:39.638164 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.638127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4zgt\" (UniqueName: \"kubernetes.io/projected/5f382bf7-d8b6-40c6-82a5-a2d625479c41-kube-api-access-j4zgt\") pod \"postgres-69fb4594cf-xr4gw\" (UID: \"5f382bf7-d8b6-40c6-82a5-a2d625479c41\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.638335 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.638199    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f382bf7-d8b6-40c6-82a5-a2d625479c41-data\") pod \"postgres-69fb4594cf-xr4gw\" (UID: \"5f382bf7-d8b6-40c6-82a5-a2d625479c41\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.739695 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.739631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f382bf7-d8b6-40c6-82a5-a2d625479c41-data\") pod \"postgres-69fb4594cf-xr4gw\" (UID: \"5f382bf7-d8b6-40c6-82a5-a2d625479c41\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.739874 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.739719    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4zgt\" (UniqueName: \"kubernetes.io/projected/5f382bf7-d8b6-40c6-82a5-a2d625479c41-kube-api-access-j4zgt\") pod \"postgres-69fb4594cf-xr4gw\" (UID: \"5f382bf7-d8b6-40c6-82a5-a2d625479c41\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.740101 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.740079    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f382bf7-d8b6-40c6-82a5-a2d625479c41-data\") pod \"postgres-69fb4594cf-xr4gw\" (UID: \"5f382bf7-d8b6-40c6-82a5-a2d625479c41\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.748506 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.748450    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4zgt\" (UniqueName: \"kubernetes.io/projected/5f382bf7-d8b6-40c6-82a5-a2d625479c41-kube-api-access-j4zgt\") pod \"postgres-69fb4594cf-xr4gw\" (UID: \"5f382bf7-d8b6-40c6-82a5-a2d625479c41\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.810319 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.810287    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:39.937136 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:39.937101    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"]
Jul 02 19:12:39.940261 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:12:39.940233    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f382bf7_d8b6_40c6_82a5_a2d625479c41.slice/crio-3254a723db852c53860f7db21def3eb0edb2cf29b08594db586e2c3a8fb00e8e WatchSource:0}: Error finding container 3254a723db852c53860f7db21def3eb0edb2cf29b08594db586e2c3a8fb00e8e: Status 404 returned error can't find the container with id 3254a723db852c53860f7db21def3eb0edb2cf29b08594db586e2c3a8fb00e8e
Jul 02 19:12:40.855583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:40.855504    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw" event={"ID":"5f382bf7-d8b6-40c6-82a5-a2d625479c41","Type":"ContainerStarted","Data":"3254a723db852c53860f7db21def3eb0edb2cf29b08594db586e2c3a8fb00e8e"}
Jul 02 19:12:45.356119 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:45.356095    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 02 19:12:45.883505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:45.883466    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw" event={"ID":"5f382bf7-d8b6-40c6-82a5-a2d625479c41","Type":"ContainerStarted","Data":"714bd95be58ff620da16d060c837841b8057811fae751603378ee5e1ebcce603"}
Jul 02 19:12:45.883681 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:45.883602    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:45.901354 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:45.901290    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw" podStartSLOduration=1.48965325 podStartE2EDuration="6.901275361s" totalImagesPullingTime="5.411622111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:45.899825068 +0000 UTC m=+545.754756864" watchObservedRunningTime="2026-07-02 19:12:45.901275361 +0000 UTC m=+545.756207130"
Jul 02 19:12:51.916244 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:51.916215    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xr4gw"
Jul 02 19:12:52.459475 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.459443    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whdr8"]
Jul 02 19:12:52.465524 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.465502    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:52.467792 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.467760    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 02 19:12:52.467924 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.467874    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m5cfg\""
Jul 02 19:12:52.467924 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.467879    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 02 19:12:52.470726 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.470696    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whdr8"]
Jul 02 19:12:52.542715 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.542679    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc7k5\" (UniqueName: \"kubernetes.io/projected/1b690958-8055-42bf-a4c0-1fe7c126cab8-kube-api-access-gc7k5\") pod \"authorino-5d4b67c446-whdr8\" (UID: \"1b690958-8055-42bf-a4c0-1fe7c126cab8\") " pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:52.643681 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.643646    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc7k5\" (UniqueName: \"kubernetes.io/projected/1b690958-8055-42bf-a4c0-1fe7c126cab8-kube-api-access-gc7k5\") pod \"authorino-5d4b67c446-whdr8\" (UID: \"1b690958-8055-42bf-a4c0-1fe7c126cab8\") " pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:52.648889 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.648863    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whdr8"]
Jul 02 19:12:52.649090 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:12:52.649072    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gc7k5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-whdr8" podUID="1b690958-8055-42bf-a4c0-1fe7c126cab8"
Jul 02 19:12:52.652151 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.652118    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc7k5\" (UniqueName: \"kubernetes.io/projected/1b690958-8055-42bf-a4c0-1fe7c126cab8-kube-api-access-gc7k5\") pod \"authorino-5d4b67c446-whdr8\" (UID: \"1b690958-8055-42bf-a4c0-1fe7c126cab8\") " pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:52.737627 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.737548    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c8444fffd-chcss"]
Jul 02 19:12:52.741135 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.741117    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:52.743563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.743542    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 02 19:12:52.749120 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.749100    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c8444fffd-chcss"]
Jul 02 19:12:52.845056 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.845024    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qp4\" (UniqueName: \"kubernetes.io/projected/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-kube-api-access-z5qp4\") pod \"authorino-6c8444fffd-chcss\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") " pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:52.845210 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.845105    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-tls-cert\") pod \"authorino-6c8444fffd-chcss\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") " pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:52.911016 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.910980    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:52.917795 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.917774    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:52.946134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.946103    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qp4\" (UniqueName: \"kubernetes.io/projected/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-kube-api-access-z5qp4\") pod \"authorino-6c8444fffd-chcss\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") " pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:52.946305 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.946164    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-tls-cert\") pod \"authorino-6c8444fffd-chcss\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") " pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:52.948645 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.948620    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-tls-cert\") pod \"authorino-6c8444fffd-chcss\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") " pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:52.956508 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:52.956481    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qp4\" (UniqueName: \"kubernetes.io/projected/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-kube-api-access-z5qp4\") pod \"authorino-6c8444fffd-chcss\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") " pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:53.047466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.047377    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc7k5\" (UniqueName: \"kubernetes.io/projected/1b690958-8055-42bf-a4c0-1fe7c126cab8-kube-api-access-gc7k5\") pod \"1b690958-8055-42bf-a4c0-1fe7c126cab8\" (UID: \"1b690958-8055-42bf-a4c0-1fe7c126cab8\") "
Jul 02 19:12:53.049388 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.049365    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b690958-8055-42bf-a4c0-1fe7c126cab8-kube-api-access-gc7k5" (OuterVolumeSpecName: "kube-api-access-gc7k5") pod "1b690958-8055-42bf-a4c0-1fe7c126cab8" (UID: "1b690958-8055-42bf-a4c0-1fe7c126cab8"). InnerVolumeSpecName "kube-api-access-gc7k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:12:53.050440 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.050419    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:12:53.148490 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.148451    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc7k5\" (UniqueName: \"kubernetes.io/projected/1b690958-8055-42bf-a4c0-1fe7c126cab8-kube-api-access-gc7k5\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:53.176448 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.176420    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c8444fffd-chcss"]
Jul 02 19:12:53.178818 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:12:53.178778    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f756d55_c97e_42b3_9b3d_9c451b28dcbd.slice/crio-3b6fccfc27fe422d23b2814ed7dbfeb27efe3751ecba0a72507cb566b85f99e5 WatchSource:0}: Error finding container 3b6fccfc27fe422d23b2814ed7dbfeb27efe3751ecba0a72507cb566b85f99e5: Status 404 returned error can't find the container with id 3b6fccfc27fe422d23b2814ed7dbfeb27efe3751ecba0a72507cb566b85f99e5
Jul 02 19:12:53.916704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.916613    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whdr8"
Jul 02 19:12:53.916704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.916644    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c8444fffd-chcss" event={"ID":"0f756d55-c97e-42b3-9b3d-9c451b28dcbd","Type":"ContainerStarted","Data":"727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522"}
Jul 02 19:12:53.916704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.916679    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c8444fffd-chcss" event={"ID":"0f756d55-c97e-42b3-9b3d-9c451b28dcbd","Type":"ContainerStarted","Data":"3b6fccfc27fe422d23b2814ed7dbfeb27efe3751ecba0a72507cb566b85f99e5"}
Jul 02 19:12:53.936208 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.936163    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c8444fffd-chcss" podStartSLOduration=1.52181444 podStartE2EDuration="1.936146529s" totalImagesPullingTime="414.332089ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:53.935803556 +0000 UTC m=+553.790735326" watchObservedRunningTime="2026-07-02 19:12:53.936146529 +0000 UTC m=+553.791078301"
Jul 02 19:12:53.963467 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.963441    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whdr8"]
Jul 02 19:12:53.967547 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:53.967525    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whdr8"]
Jul 02 19:12:54.776793 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:54.776761    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b690958-8055-42bf-a4c0-1fe7c126cab8" path="/var/lib/kubelet/pods/1b690958-8055-42bf-a4c0-1fe7c126cab8/volumes"
Jul 02 19:12:58.381918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.381883    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-656f8b85bd-2x45d"]
Jul 02 19:12:58.625335 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.625301    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656f8b85bd-2x45d"]
Jul 02 19:12:58.625522 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.625367    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6464bbbddb-88dk4"]
Jul 02 19:12:58.625522 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.625441    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.628573 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.628294    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 02 19:12:58.628573 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.628406    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lzbkb\""
Jul 02 19:12:58.628573 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.628294    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 02 19:12:58.651962 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.651932    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-88dk4"]
Jul 02 19:12:58.652104 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.652032    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.797609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.797581    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldqp2\" (UniqueName: \"kubernetes.io/projected/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-kube-api-access-ldqp2\") pod \"maas-controller-656f8b85bd-2x45d\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") " pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.797774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.797643    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-webhook-cert\") pod \"maas-controller-656f8b85bd-2x45d\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") " pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.797774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.797683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7h8c\" (UniqueName: \"kubernetes.io/projected/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-kube-api-access-h7h8c\") pod \"maas-controller-6464bbbddb-88dk4\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") " pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.797774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.797707    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-webhook-cert\") pod \"maas-controller-6464bbbddb-88dk4\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") " pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.899038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.898949    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7h8c\" (UniqueName: \"kubernetes.io/projected/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-kube-api-access-h7h8c\") pod \"maas-controller-6464bbbddb-88dk4\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") " pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.899038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.899001    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-webhook-cert\") pod \"maas-controller-6464bbbddb-88dk4\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") " pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.899038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.899024    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldqp2\" (UniqueName: \"kubernetes.io/projected/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-kube-api-access-ldqp2\") pod \"maas-controller-656f8b85bd-2x45d\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") " pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.899302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.899133    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-webhook-cert\") pod \"maas-controller-656f8b85bd-2x45d\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") " pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.901690 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.901660    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-webhook-cert\") pod \"maas-controller-656f8b85bd-2x45d\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") " pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.901806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.901693    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-webhook-cert\") pod \"maas-controller-6464bbbddb-88dk4\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") " pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.907628 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.907600    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7h8c\" (UniqueName: \"kubernetes.io/projected/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-kube-api-access-h7h8c\") pod \"maas-controller-6464bbbddb-88dk4\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") " pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:58.907772 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.907670    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldqp2\" (UniqueName: \"kubernetes.io/projected/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-kube-api-access-ldqp2\") pod \"maas-controller-656f8b85bd-2x45d\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") " pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.940844 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.940814    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:12:58.960995 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:58.960967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:12:59.112389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:59.112271    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656f8b85bd-2x45d"]
Jul 02 19:12:59.115606 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:12:59.115578    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b4a95c_d02e_4fec_abf2_c4957c3ca562.slice/crio-bedaac7b5850649ded4c88eecdce9957043077a4dacd7c392d5c70d1329a342f WatchSource:0}: Error finding container bedaac7b5850649ded4c88eecdce9957043077a4dacd7c392d5c70d1329a342f: Status 404 returned error can't find the container with id bedaac7b5850649ded4c88eecdce9957043077a4dacd7c392d5c70d1329a342f
Jul 02 19:12:59.137586 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:59.137562    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-88dk4"]
Jul 02 19:12:59.140189 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:12:59.140163    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b37f6d_5d8e_4ebd_b1a9_edfcc1b7ab25.slice/crio-258e9f37d7a919ef5652487d3db5536216e0ab7b2e163bc8ec1943411ed0f09b WatchSource:0}: Error finding container 258e9f37d7a919ef5652487d3db5536216e0ab7b2e163bc8ec1943411ed0f09b: Status 404 returned error can't find the container with id 258e9f37d7a919ef5652487d3db5536216e0ab7b2e163bc8ec1943411ed0f09b
Jul 02 19:12:59.946641 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:59.946560    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6464bbbddb-88dk4" event={"ID":"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25","Type":"ContainerStarted","Data":"258e9f37d7a919ef5652487d3db5536216e0ab7b2e163bc8ec1943411ed0f09b"}
Jul 02 19:12:59.948292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:12:59.948262    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656f8b85bd-2x45d" event={"ID":"f4b4a95c-d02e-4fec-abf2-c4957c3ca562","Type":"ContainerStarted","Data":"bedaac7b5850649ded4c88eecdce9957043077a4dacd7c392d5c70d1329a342f"}
Jul 02 19:13:01.958430 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:01.958391    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6464bbbddb-88dk4" event={"ID":"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25","Type":"ContainerStarted","Data":"e2755a75960246bed6af33afd5a8da04d6c675df1e84c23f468f5fc61ba1c174"}
Jul 02 19:13:01.958875 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:01.958482    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:13:01.960090 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:01.960068    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656f8b85bd-2x45d" event={"ID":"f4b4a95c-d02e-4fec-abf2-c4957c3ca562","Type":"ContainerStarted","Data":"89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb"}
Jul 02 19:13:01.960172 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:01.960115    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:13:01.977534 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:01.977470    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6464bbbddb-88dk4" podStartSLOduration=1.8641005069999999 podStartE2EDuration="3.977453229s" totalImagesPullingTime="2.113352722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:01.977131026 +0000 UTC m=+561.832062797" watchObservedRunningTime="2026-07-02 19:13:01.977453229 +0000 UTC m=+561.832385004"
Jul 02 19:13:01.995985 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:01.995943    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-656f8b85bd-2x45d" podStartSLOduration=1.8488029940000001 podStartE2EDuration="3.995927669s" totalImagesPullingTime="2.147124675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:01.994861754 +0000 UTC m=+561.849793543" watchObservedRunningTime="2026-07-02 19:13:01.995927669 +0000 UTC m=+561.850859476"
Jul 02 19:13:12.968975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:12.968942    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:13:12.969462 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:12.969024    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:13:13.028932 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.028903    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-656f8b85bd-2x45d"]
Jul 02 19:13:13.029111 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.029087    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-656f8b85bd-2x45d" podUID="f4b4a95c-d02e-4fec-abf2-c4957c3ca562" containerName="manager" containerID="cri-o://89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb" gracePeriod=30
Jul 02 19:13:13.268051 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.268022    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:13:13.319813 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.319783    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-webhook-cert\") pod \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") "
Jul 02 19:13:13.319961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.319883    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldqp2\" (UniqueName: \"kubernetes.io/projected/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-kube-api-access-ldqp2\") pod \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\" (UID: \"f4b4a95c-d02e-4fec-abf2-c4957c3ca562\") "
Jul 02 19:13:13.321925 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.321899    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f4b4a95c-d02e-4fec-abf2-c4957c3ca562" (UID: "f4b4a95c-d02e-4fec-abf2-c4957c3ca562"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:13:13.322039 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.321953    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-kube-api-access-ldqp2" (OuterVolumeSpecName: "kube-api-access-ldqp2") pod "f4b4a95c-d02e-4fec-abf2-c4957c3ca562" (UID: "f4b4a95c-d02e-4fec-abf2-c4957c3ca562"). InnerVolumeSpecName "kube-api-access-ldqp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:13:13.420964 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.420929    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-webhook-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:13:13.420964 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:13.420957    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldqp2\" (UniqueName: \"kubernetes.io/projected/f4b4a95c-d02e-4fec-abf2-c4957c3ca562-kube-api-access-ldqp2\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:13:14.008916 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.008882    2584 generic.go:356] "Generic (PLEG): container finished" podID="f4b4a95c-d02e-4fec-abf2-c4957c3ca562" containerID="89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb" exitCode=0
Jul 02 19:13:14.009321 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.008928    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656f8b85bd-2x45d"
Jul 02 19:13:14.009321 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.008955    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656f8b85bd-2x45d" event={"ID":"f4b4a95c-d02e-4fec-abf2-c4957c3ca562","Type":"ContainerDied","Data":"89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb"}
Jul 02 19:13:14.009321 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.008993    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656f8b85bd-2x45d" event={"ID":"f4b4a95c-d02e-4fec-abf2-c4957c3ca562","Type":"ContainerDied","Data":"bedaac7b5850649ded4c88eecdce9957043077a4dacd7c392d5c70d1329a342f"}
Jul 02 19:13:14.009321 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.009014    2584 scope.go:117] "RemoveContainer" containerID="89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb"
Jul 02 19:13:14.017585 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.017568    2584 scope.go:117] "RemoveContainer" containerID="89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb"
Jul 02 19:13:14.017827 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:13:14.017807    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb\": container with ID starting with 89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb not found: ID does not exist" containerID="89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb"
Jul 02 19:13:14.017864 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.017835    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb"} err="failed to get container status \"89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb\": rpc error: code = NotFound desc = could not find container \"89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb\": container with ID starting with 89f892d0da81b7b533dd768a26703471ab0e1ea26fa4007dcfad0f066480d4cb not found: ID does not exist"
Jul 02 19:13:14.029707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.029684    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-656f8b85bd-2x45d"]
Jul 02 19:13:14.033289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.033269    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-656f8b85bd-2x45d"]
Jul 02 19:13:14.776770 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:14.776737    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b4a95c-d02e-4fec-abf2-c4957c3ca562" path="/var/lib/kubelet/pods/f4b4a95c-d02e-4fec-abf2-c4957c3ca562/volumes"
Jul 02 19:13:33.392844 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.392792    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"]
Jul 02 19:13:33.393267 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.393253    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b4a95c-d02e-4fec-abf2-c4957c3ca562" containerName="manager"
Jul 02 19:13:33.393310 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.393270    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b4a95c-d02e-4fec-abf2-c4957c3ca562" containerName="manager"
Jul 02 19:13:33.393388 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.393375    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4b4a95c-d02e-4fec-abf2-c4957c3ca562" containerName="manager"
Jul 02 19:13:33.396177 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.396160    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.398746 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.398714    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g6bdc\""
Jul 02 19:13:33.398881 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.398809    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 02 19:13:33.427394 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.427368    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"]
Jul 02 19:13:33.490712 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.490683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87440443-6517-451e-8738-0980b80ff454-openshift-service-ca\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.490873 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.490716    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87440443-6517-451e-8738-0980b80ff454-maas-api-tls\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.490873 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.490752    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgrxs\" (UniqueName: \"kubernetes.io/projected/87440443-6517-451e-8738-0980b80ff454-kube-api-access-jgrxs\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.591309 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.591268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87440443-6517-451e-8738-0980b80ff454-openshift-service-ca\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.591309 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.591309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87440443-6517-451e-8738-0980b80ff454-maas-api-tls\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.591547 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.591334    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgrxs\" (UniqueName: \"kubernetes.io/projected/87440443-6517-451e-8738-0980b80ff454-kube-api-access-jgrxs\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.591953 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.591931    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87440443-6517-451e-8738-0980b80ff454-openshift-service-ca\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.593900 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.593874    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87440443-6517-451e-8738-0980b80ff454-maas-api-tls\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.600512 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.600485    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgrxs\" (UniqueName: \"kubernetes.io/projected/87440443-6517-451e-8738-0980b80ff454-kube-api-access-jgrxs\") pod \"maas-api-6d7bb6898d-bk7dt\" (UID: \"87440443-6517-451e-8738-0980b80ff454\") " pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.705709 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.705618    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:33.857360 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:33.857317    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"]
Jul 02 19:13:33.860411 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:13:33.860380    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87440443_6517_451e_8738_0980b80ff454.slice/crio-6559f1f09bd16922bc77e4654106b260fe48330ed9020825ab4b45ca9f70cdf4 WatchSource:0}: Error finding container 6559f1f09bd16922bc77e4654106b260fe48330ed9020825ab4b45ca9f70cdf4: Status 404 returned error can't find the container with id 6559f1f09bd16922bc77e4654106b260fe48330ed9020825ab4b45ca9f70cdf4
Jul 02 19:13:34.095051 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.095015    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt" event={"ID":"87440443-6517-451e-8738-0980b80ff454","Type":"ContainerStarted","Data":"6559f1f09bd16922bc77e4654106b260fe48330ed9020825ab4b45ca9f70cdf4"}
Jul 02 19:13:34.457253 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.457174    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ddfm5"]
Jul 02 19:13:34.459292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.459276    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.463836 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.463816    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g4gvv\""
Jul 02 19:13:34.463961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.463816    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 02 19:13:34.476390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.476362    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ddfm5"]
Jul 02 19:13:34.600814 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.600586    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6p9c\" (UniqueName: \"kubernetes.io/projected/78211ab9-79b8-43d1-902a-fd8b7cb524e0-kube-api-access-x6p9c\") pod \"payload-processing-58f86c7798-ddfm5\" (UID: \"78211ab9-79b8-43d1-902a-fd8b7cb524e0\") " pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.600814 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.600726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78211ab9-79b8-43d1-902a-fd8b7cb524e0-plugins-config-volume\") pod \"payload-processing-58f86c7798-ddfm5\" (UID: \"78211ab9-79b8-43d1-902a-fd8b7cb524e0\") " pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.702043 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.702011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6p9c\" (UniqueName: \"kubernetes.io/projected/78211ab9-79b8-43d1-902a-fd8b7cb524e0-kube-api-access-x6p9c\") pod \"payload-processing-58f86c7798-ddfm5\" (UID: \"78211ab9-79b8-43d1-902a-fd8b7cb524e0\") " pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.702229 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.702081    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78211ab9-79b8-43d1-902a-fd8b7cb524e0-plugins-config-volume\") pod \"payload-processing-58f86c7798-ddfm5\" (UID: \"78211ab9-79b8-43d1-902a-fd8b7cb524e0\") " pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.702790 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.702772    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78211ab9-79b8-43d1-902a-fd8b7cb524e0-plugins-config-volume\") pod \"payload-processing-58f86c7798-ddfm5\" (UID: \"78211ab9-79b8-43d1-902a-fd8b7cb524e0\") " pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.715212 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.715148    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6p9c\" (UniqueName: \"kubernetes.io/projected/78211ab9-79b8-43d1-902a-fd8b7cb524e0-kube-api-access-x6p9c\") pod \"payload-processing-58f86c7798-ddfm5\" (UID: \"78211ab9-79b8-43d1-902a-fd8b7cb524e0\") " pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.768802 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.768772    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:34.938382 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:34.938354    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ddfm5"]
Jul 02 19:13:34.939700 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:13:34.939675    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78211ab9_79b8_43d1_902a_fd8b7cb524e0.slice/crio-4b8d0e036e5584fe49ecc01ad4fabc8be8ace1b103f99975ad5b41918610ffd1 WatchSource:0}: Error finding container 4b8d0e036e5584fe49ecc01ad4fabc8be8ace1b103f99975ad5b41918610ffd1: Status 404 returned error can't find the container with id 4b8d0e036e5584fe49ecc01ad4fabc8be8ace1b103f99975ad5b41918610ffd1
Jul 02 19:13:35.101667 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:35.101626    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5" event={"ID":"78211ab9-79b8-43d1-902a-fd8b7cb524e0","Type":"ContainerStarted","Data":"4b8d0e036e5584fe49ecc01ad4fabc8be8ace1b103f99975ad5b41918610ffd1"}
Jul 02 19:13:36.108149 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:36.108118    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt" event={"ID":"87440443-6517-451e-8738-0980b80ff454","Type":"ContainerStarted","Data":"318c863fb9be6a8359ee900817915a56e45cbf5adc2c9ce5914b8d1cb7eead01"}
Jul 02 19:13:36.108558 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:36.108208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:36.125134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:36.125083    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt" podStartSLOduration=1.460996577 podStartE2EDuration="3.125068691s" totalImagesPullingTime="1.664072114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:36.124687145 +0000 UTC m=+595.979618924" watchObservedRunningTime="2026-07-02 19:13:36.125068691 +0000 UTC m=+595.980000460"
Jul 02 19:13:37.114119 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:37.114086    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5" event={"ID":"78211ab9-79b8-43d1-902a-fd8b7cb524e0","Type":"ContainerStarted","Data":"abde27e38fe95a2f58ff7429edf270c1a4436e506df8dbadbfe81ddff1df6b5d"}
Jul 02 19:13:37.133049 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:37.132986    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5" podStartSLOduration=1.492343754 podStartE2EDuration="3.132951493s" totalImagesPullingTime="1.640607739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:37.130214702 +0000 UTC m=+596.985146497" watchObservedRunningTime="2026-07-02 19:13:37.132951493 +0000 UTC m=+596.987883265"
Jul 02 19:13:38.119087 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:38.119053    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:42.119978 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.119949    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d7bb6898d-bk7dt"
Jul 02 19:13:42.139094 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.139066    2584 generic.go:356] "Generic (PLEG): container finished" podID="da9439cf-a90e-4a70-9b66-5ccb6f5feb9e" containerID="c088f853c837ea3d56a594fbd1a5ca276e4980dcb64e87cb8237c2a965b4f847" exitCode=0
Jul 02 19:13:42.139230 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.139104    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" event={"ID":"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e","Type":"ContainerDied","Data":"c088f853c837ea3d56a594fbd1a5ca276e4980dcb64e87cb8237c2a965b4f847"}
Jul 02 19:13:42.139465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.139447    2584 scope.go:117] "RemoveContainer" containerID="c088f853c837ea3d56a594fbd1a5ca276e4980dcb64e87cb8237c2a965b4f847"
Jul 02 19:13:42.140498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.140457    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:13:42.140576 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.140541    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:13:42.140627 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:42.140586    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:13:43.149328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:43.149285    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" event={"ID":"da9439cf-a90e-4a70-9b66-5ccb6f5feb9e","Type":"ContainerStarted","Data":"9d49c0a8abacc4c6771b45530a4977e55721814d7d4a3290d88b592e59b1c997"}
Jul 02 19:13:43.443935 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:43.443835    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:13:43.444116 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:43.444002    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-5h8rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 02 19:13:43.444116 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:43.444049    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" podUID="da9439cf-a90e-4a70-9b66-5ccb6f5feb9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 02 19:13:44.443046 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:44.442997    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-5h8rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 02 19:13:44.443557 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:44.443070    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" podUID="da9439cf-a90e-4a70-9b66-5ccb6f5feb9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 02 19:13:45.443319 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:45.443272    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-5h8rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 02 19:13:45.443812 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:45.443363    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" podUID="da9439cf-a90e-4a70-9b66-5ccb6f5feb9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 02 19:13:46.442631 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:46.442588    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-5h8rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 02 19:13:46.442930 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:46.442899    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj" podUID="da9439cf-a90e-4a70-9b66-5ccb6f5feb9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 02 19:13:47.446406 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:47.446377    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:13:47.446779 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:47.446666    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:13:47.447391 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:47.447376    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-5h8rj"
Jul 02 19:13:49.124723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:49.124693    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-ddfm5"
Jul 02 19:13:51.330871 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.330841    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"]
Jul 02 19:13:51.333357 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.333327    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.337014 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.336987    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 02 19:13:51.338136 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.338112    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 02 19:13:51.338241 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.338134    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 02 19:13:51.338328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.338256    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t76xx\""
Jul 02 19:13:51.345640 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.345622    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"]
Jul 02 19:13:51.448747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.448708    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da319c4-0364-48a6-a6c2-082b9bf029da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.448912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.448761    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnk9f\" (UniqueName: \"kubernetes.io/projected/6da319c4-0364-48a6-a6c2-082b9bf029da-kube-api-access-vnk9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.448912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.448845    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.449035 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.448930    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.449035 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.448969    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.449035 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.449019    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549433 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549449    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549469    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549518    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da319c4-0364-48a6-a6c2-082b9bf029da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnk9f\" (UniqueName: \"kubernetes.io/projected/6da319c4-0364-48a6-a6c2-082b9bf029da-kube-api-access-vnk9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.549929 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549901    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.550033 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549897    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.550033 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.549970    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.551746 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.551728    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da319c4-0364-48a6-a6c2-082b9bf029da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.551925 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.551907    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da319c4-0364-48a6-a6c2-082b9bf029da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.558470 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.558444    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnk9f\" (UniqueName: \"kubernetes.io/projected/6da319c4-0364-48a6-a6c2-082b9bf029da-kube-api-access-vnk9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2\" (UID: \"6da319c4-0364-48a6-a6c2-082b9bf029da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.643420 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.643323    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:13:51.771882 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:51.771858    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"]
Jul 02 19:13:51.773803 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:13:51.773774    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da319c4_0364_48a6_a6c2_082b9bf029da.slice/crio-f22a4e84785173c564da5db2fd4ceff4d8f2e2e1ab2273d4af3a51b789a706c0 WatchSource:0}: Error finding container f22a4e84785173c564da5db2fd4ceff4d8f2e2e1ab2273d4af3a51b789a706c0: Status 404 returned error can't find the container with id f22a4e84785173c564da5db2fd4ceff4d8f2e2e1ab2273d4af3a51b789a706c0
Jul 02 19:13:52.195216 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:52.195125    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2" event={"ID":"6da319c4-0364-48a6-a6c2-082b9bf029da","Type":"ContainerStarted","Data":"f22a4e84785173c564da5db2fd4ceff4d8f2e2e1ab2273d4af3a51b789a706c0"}
Jul 02 19:13:53.200509 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:53.200426    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2" event={"ID":"6da319c4-0364-48a6-a6c2-082b9bf029da","Type":"ContainerStarted","Data":"c63be959c829b6d530d5e52e9be44cf723382e931c493ba0344d25eafddf01c2"}
Jul 02 19:13:53.216770 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:53.216718    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2" podStartSLOduration=1.069485482 podStartE2EDuration="2.216665986s" totalImagesPullingTime="1.147180504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:53.216490021 +0000 UTC m=+613.071421793" watchObservedRunningTime="2026-07-02 19:13:53.216665986 +0000 UTC m=+613.071597756"
Jul 02 19:13:59.532548 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.532508    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"]
Jul 02 19:13:59.534853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.534829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.537161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.537142    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 02 19:13:59.546437 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.546411    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"]
Jul 02 19:13:59.725234 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.725194    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.725452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.725260    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn6wq\" (UniqueName: \"kubernetes.io/projected/53c7546d-602a-4336-85e9-0ba992e1fc48-kube-api-access-gn6wq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.725452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.725303    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.725452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.725358    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.725452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.725396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.725653 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.725457    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c7546d-602a-4336-85e9-0ba992e1fc48-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.826903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.826818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.826903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.826874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn6wq\" (UniqueName: \"kubernetes.io/projected/53c7546d-602a-4336-85e9-0ba992e1fc48-kube-api-access-gn6wq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827122 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.826909    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827122 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.826937    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827122 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.826965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827122 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.826987    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c7546d-602a-4336-85e9-0ba992e1fc48-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827326 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.827199    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827326 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.827298    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.827453 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.827392    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.829101 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.829081    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c7546d-602a-4336-85e9-0ba992e1fc48-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.829478 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.829458    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c7546d-602a-4336-85e9-0ba992e1fc48-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.835393 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.835371    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn6wq\" (UniqueName: \"kubernetes.io/projected/53c7546d-602a-4336-85e9-0ba992e1fc48-kube-api-access-gn6wq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jpk2z\" (UID: \"53c7546d-602a-4336-85e9-0ba992e1fc48\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.844888 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.844863    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:13:59.975161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:13:59.975120    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"]
Jul 02 19:13:59.977500 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:13:59.977469    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c7546d_602a_4336_85e9_0ba992e1fc48.slice/crio-09472664fc3bf1c8fcdb8038c05b302da8fbe97e41a919b415e91ec1f8a9b843 WatchSource:0}: Error finding container 09472664fc3bf1c8fcdb8038c05b302da8fbe97e41a919b415e91ec1f8a9b843: Status 404 returned error can't find the container with id 09472664fc3bf1c8fcdb8038c05b302da8fbe97e41a919b415e91ec1f8a9b843
Jul 02 19:14:00.236076 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:00.235954    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z" event={"ID":"53c7546d-602a-4336-85e9-0ba992e1fc48","Type":"ContainerStarted","Data":"09472664fc3bf1c8fcdb8038c05b302da8fbe97e41a919b415e91ec1f8a9b843"}
Jul 02 19:14:01.241940 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:01.241903    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z" event={"ID":"53c7546d-602a-4336-85e9-0ba992e1fc48","Type":"ContainerStarted","Data":"d0ec4c68e22c07e21316bbbdfa5bfade9464eadbfb60b60193cfb699147a4a1e"}
Jul 02 19:14:01.269118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:01.269071    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z" podStartSLOduration=2.059420861 podStartE2EDuration="2.269055026s" totalImagesPullingTime="209.634165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:01.266541659 +0000 UTC m=+621.121473448" watchObservedRunningTime="2026-07-02 19:14:01.269055026 +0000 UTC m=+621.123986795"
Jul 02 19:14:01.643978 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:01.643947    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:14:01.644191 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:01.644005    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:14:01.656171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:01.656143    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:14:02.257432 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:02.257353    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2"
Jul 02 19:14:09.845076 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:09.845036    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:14:09.845573 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:09.845113    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:14:09.857261 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:09.857241    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:14:10.292123 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:14:10.292046    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jpk2z"
Jul 02 19:16:12.779289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:12.779255    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c8444fffd-chcss"]
Jul 02 19:16:12.779860 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:12.779455    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6c8444fffd-chcss" podUID="0f756d55-c97e-42b3-9b3d-9c451b28dcbd" containerName="authorino" containerID="cri-o://727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522" gracePeriod=30
Jul 02 19:16:13.036053 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.035994    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:16:13.143924 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.143878    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5qp4\" (UniqueName: \"kubernetes.io/projected/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-kube-api-access-z5qp4\") pod \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") "
Jul 02 19:16:13.143924 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.143938    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-tls-cert\") pod \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\" (UID: \"0f756d55-c97e-42b3-9b3d-9c451b28dcbd\") "
Jul 02 19:16:13.146033 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.145998    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-kube-api-access-z5qp4" (OuterVolumeSpecName: "kube-api-access-z5qp4") pod "0f756d55-c97e-42b3-9b3d-9c451b28dcbd" (UID: "0f756d55-c97e-42b3-9b3d-9c451b28dcbd"). InnerVolumeSpecName "kube-api-access-z5qp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:16:13.155580 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.155550    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0f756d55-c97e-42b3-9b3d-9c451b28dcbd" (UID: "0f756d55-c97e-42b3-9b3d-9c451b28dcbd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:16:13.245081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.245047    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5qp4\" (UniqueName: \"kubernetes.io/projected/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-kube-api-access-z5qp4\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:16:13.245081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.245074    2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f756d55-c97e-42b3-9b3d-9c451b28dcbd-tls-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:16:13.827276 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.827239    2584 generic.go:356] "Generic (PLEG): container finished" podID="0f756d55-c97e-42b3-9b3d-9c451b28dcbd" containerID="727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522" exitCode=0
Jul 02 19:16:13.827636 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.827295    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c8444fffd-chcss"
Jul 02 19:16:13.827636 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.827322    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c8444fffd-chcss" event={"ID":"0f756d55-c97e-42b3-9b3d-9c451b28dcbd","Type":"ContainerDied","Data":"727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522"}
Jul 02 19:16:13.827636 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.827394    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c8444fffd-chcss" event={"ID":"0f756d55-c97e-42b3-9b3d-9c451b28dcbd","Type":"ContainerDied","Data":"3b6fccfc27fe422d23b2814ed7dbfeb27efe3751ecba0a72507cb566b85f99e5"}
Jul 02 19:16:13.827636 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.827416    2584 scope.go:117] "RemoveContainer" containerID="727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522"
Jul 02 19:16:13.836325 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.836301    2584 scope.go:117] "RemoveContainer" containerID="727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522"
Jul 02 19:16:13.836630 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:16:13.836610    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522\": container with ID starting with 727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522 not found: ID does not exist" containerID="727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522"
Jul 02 19:16:13.836693 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.836638    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522"} err="failed to get container status \"727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522\": rpc error: code = NotFound desc = could not find container \"727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522\": container with ID starting with 727641be2fd4ec28bc9fb5369f90c0c80334ee4d3b812321fff008b0edc83522 not found: ID does not exist"
Jul 02 19:16:13.852117 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.852090    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c8444fffd-chcss"]
Jul 02 19:16:13.861230 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:13.861206    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c8444fffd-chcss"]
Jul 02 19:16:14.777214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:16:14.777179    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f756d55-c97e-42b3-9b3d-9c451b28dcbd" path="/var/lib/kubelet/pods/0f756d55-c97e-42b3-9b3d-9c451b28dcbd/volumes"
Jul 02 19:17:24.977215 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:24.977182    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-88dk4"]
Jul 02 19:17:24.977730 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:24.977451    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6464bbbddb-88dk4" podUID="12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" containerName="manager" containerID="cri-o://e2755a75960246bed6af33afd5a8da04d6c675df1e84c23f468f5fc61ba1c174" gracePeriod=30
Jul 02 19:17:25.139115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.139084    2584 generic.go:356] "Generic (PLEG): container finished" podID="12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" containerID="e2755a75960246bed6af33afd5a8da04d6c675df1e84c23f468f5fc61ba1c174" exitCode=0
Jul 02 19:17:25.139287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.139156    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6464bbbddb-88dk4" event={"ID":"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25","Type":"ContainerDied","Data":"e2755a75960246bed6af33afd5a8da04d6c675df1e84c23f468f5fc61ba1c174"}
Jul 02 19:17:25.223105 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.223075    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:17:25.366231 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.366198    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-webhook-cert\") pod \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") "
Jul 02 19:17:25.366417 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.366252    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7h8c\" (UniqueName: \"kubernetes.io/projected/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-kube-api-access-h7h8c\") pod \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\" (UID: \"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25\") "
Jul 02 19:17:25.368363 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.368319    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-kube-api-access-h7h8c" (OuterVolumeSpecName: "kube-api-access-h7h8c") pod "12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" (UID: "12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25"). InnerVolumeSpecName "kube-api-access-h7h8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:17:25.368449 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.368331    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" (UID: "12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:17:25.467638 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.467612    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-webhook-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:17:25.467638 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:25.467638    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7h8c\" (UniqueName: \"kubernetes.io/projected/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25-kube-api-access-h7h8c\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:17:26.145274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.145245    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6464bbbddb-88dk4"
Jul 02 19:17:26.145689 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.145245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6464bbbddb-88dk4" event={"ID":"12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25","Type":"ContainerDied","Data":"258e9f37d7a919ef5652487d3db5536216e0ab7b2e163bc8ec1943411ed0f09b"}
Jul 02 19:17:26.145689 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.145388    2584 scope.go:117] "RemoveContainer" containerID="e2755a75960246bed6af33afd5a8da04d6c675df1e84c23f468f5fc61ba1c174"
Jul 02 19:17:26.165630 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.165604    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-88dk4"]
Jul 02 19:17:26.169244 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.169223    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-88dk4"]
Jul 02 19:17:26.776289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.776254    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" path="/var/lib/kubelet/pods/12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25/volumes"
Jul 02 19:17:26.952451 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952421    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6464bbbddb-g2vth"]
Jul 02 19:17:26.952745 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952733    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" containerName="manager"
Jul 02 19:17:26.952795 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952748    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" containerName="manager"
Jul 02 19:17:26.952795 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952760    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f756d55-c97e-42b3-9b3d-9c451b28dcbd" containerName="authorino"
Jul 02 19:17:26.952795 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952766    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f756d55-c97e-42b3-9b3d-9c451b28dcbd" containerName="authorino"
Jul 02 19:17:26.952886 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952826    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="12b37f6d-5d8e-4ebd-b1a9-edfcc1b7ab25" containerName="manager"
Jul 02 19:17:26.952886 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.952835    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f756d55-c97e-42b3-9b3d-9c451b28dcbd" containerName="authorino"
Jul 02 19:17:26.956856 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.956837    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:26.960063 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.960040    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 02 19:17:26.960176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.960065    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 02 19:17:26.960176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.960041    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lzbkb\""
Jul 02 19:17:26.965586 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:26.965564    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-g2vth"]
Jul 02 19:17:27.080150 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.080062    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-649nb\" (UniqueName: \"kubernetes.io/projected/4ce4beab-3cfb-4db3-847a-ad22302b6469-kube-api-access-649nb\") pod \"maas-controller-6464bbbddb-g2vth\" (UID: \"4ce4beab-3cfb-4db3-847a-ad22302b6469\") " pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.080150 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.080102    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ce4beab-3cfb-4db3-847a-ad22302b6469-webhook-cert\") pod \"maas-controller-6464bbbddb-g2vth\" (UID: \"4ce4beab-3cfb-4db3-847a-ad22302b6469\") " pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.181161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.181127    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-649nb\" (UniqueName: \"kubernetes.io/projected/4ce4beab-3cfb-4db3-847a-ad22302b6469-kube-api-access-649nb\") pod \"maas-controller-6464bbbddb-g2vth\" (UID: \"4ce4beab-3cfb-4db3-847a-ad22302b6469\") " pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.181161 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.181160    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ce4beab-3cfb-4db3-847a-ad22302b6469-webhook-cert\") pod \"maas-controller-6464bbbddb-g2vth\" (UID: \"4ce4beab-3cfb-4db3-847a-ad22302b6469\") " pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.183550 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.183532    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ce4beab-3cfb-4db3-847a-ad22302b6469-webhook-cert\") pod \"maas-controller-6464bbbddb-g2vth\" (UID: \"4ce4beab-3cfb-4db3-847a-ad22302b6469\") " pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.189045 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.189020    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-649nb\" (UniqueName: \"kubernetes.io/projected/4ce4beab-3cfb-4db3-847a-ad22302b6469-kube-api-access-649nb\") pod \"maas-controller-6464bbbddb-g2vth\" (UID: \"4ce4beab-3cfb-4db3-847a-ad22302b6469\") " pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.267755 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.267723    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:27.397160 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.397133    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6464bbbddb-g2vth"]
Jul 02 19:17:27.398798 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:17:27.398750    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce4beab_3cfb_4db3_847a_ad22302b6469.slice/crio-6494fbe1dbe64e7e1e1589027d0eae27ec3859d30b82e7bc3da8f1a3a64ac609 WatchSource:0}: Error finding container 6494fbe1dbe64e7e1e1589027d0eae27ec3859d30b82e7bc3da8f1a3a64ac609: Status 404 returned error can't find the container with id 6494fbe1dbe64e7e1e1589027d0eae27ec3859d30b82e7bc3da8f1a3a64ac609
Jul 02 19:17:27.400098 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:27.400080    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:17:28.155746 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:28.155717    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6464bbbddb-g2vth" event={"ID":"4ce4beab-3cfb-4db3-847a-ad22302b6469","Type":"ContainerStarted","Data":"8e0ad5debb444260f567007fb1b105d454350e9ad78a0a73895776a2afd4b442"}
Jul 02 19:17:28.155915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:28.155768    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6464bbbddb-g2vth" event={"ID":"4ce4beab-3cfb-4db3-847a-ad22302b6469","Type":"ContainerStarted","Data":"6494fbe1dbe64e7e1e1589027d0eae27ec3859d30b82e7bc3da8f1a3a64ac609"}
Jul 02 19:17:28.155915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:28.155820    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:17:28.178954 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:28.178899    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6464bbbddb-g2vth" podStartSLOduration=1.527939511 podStartE2EDuration="2.178882086s" totalImagesPullingTime="650.942575ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:17:28.175909696 +0000 UTC m=+828.030841469" watchObservedRunningTime="2026-07-02 19:17:28.178882086 +0000 UTC m=+828.033813856"
Jul 02 19:17:39.165394 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:17:39.165363    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6464bbbddb-g2vth"
Jul 02 19:29:26.354492 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.354456    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"]
Jul 02 19:29:26.357761 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.357745    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.360629 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.360609    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3ef64d22-openshift-default-dockercfg-ts25m\""
Jul 02 19:29:26.380326 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.380294    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"]
Jul 02 19:29:26.502298 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502262    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-socket\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502298 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502300    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-token\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-envoy\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502436    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-podinfo\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwjrk\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-kube-api-access-hwjrk\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502566    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-data\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502596    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-certs\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istiod-ca-cert\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.502732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.502668    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-credential-socket\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603718 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-socket\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603718 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603716    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-token\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-envoy\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603760    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-podinfo\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603777    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwjrk\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-kube-api-access-hwjrk\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603810    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-data\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603837    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-certs\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603870    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istiod-ca-cert\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.603968 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.603891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-credential-socket\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.604305 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.604190    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-socket\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.604305 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.604234    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-data\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.604305 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.604286    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-certs\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.604525 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.604317    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-credential-socket\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.604716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.604693    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istiod-ca-cert\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.606162 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.606139    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-envoy\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.606270 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.606204    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-podinfo\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.613576 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.613548    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-token\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.614701 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.614675    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwjrk\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-kube-api-access-hwjrk\") pod \"e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") " pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.668331 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.668285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:26.803643 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.803619    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"]
Jul 02 19:29:26.805384 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:29:26.805355    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16b16bc_aa2d_4ed5_b550_46385ce63b17.slice/crio-7c633136c058d19b49ffbab5998e9d1eb94471195510692378fdcae9fb0dd9ed WatchSource:0}: Error finding container 7c633136c058d19b49ffbab5998e9d1eb94471195510692378fdcae9fb0dd9ed: Status 404 returned error can't find the container with id 7c633136c058d19b49ffbab5998e9d1eb94471195510692378fdcae9fb0dd9ed
Jul 02 19:29:26.807316 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.807284    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:26.807403 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.807372    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:26.807441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:26.807402    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:27.242246 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.242153    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c" event={"ID":"f16b16bc-aa2d-4ed5-b550-46385ce63b17","Type":"ContainerStarted","Data":"2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5"}
Jul 02 19:29:27.242246 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.242207    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c" event={"ID":"f16b16bc-aa2d-4ed5-b550-46385ce63b17","Type":"ContainerStarted","Data":"7c633136c058d19b49ffbab5998e9d1eb94471195510692378fdcae9fb0dd9ed"}
Jul 02 19:29:27.266241 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.266197    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c" podStartSLOduration=1.266181755 podStartE2EDuration="1.266181755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:29:27.26445548 +0000 UTC m=+1547.119387251" watchObservedRunningTime="2026-07-02 19:29:27.266181755 +0000 UTC m=+1547.121113524"
Jul 02 19:29:27.668511 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.668474    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:27.673560 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.673531    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:27.831884 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.831853    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"]
Jul 02 19:29:27.835211 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.835194    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:27.838077 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.838056    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3ef64d22-serving-cert\""
Jul 02 19:29:27.844878 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.844847    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"]
Jul 02 19:29:27.915974 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.915940    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-openshift-service-ca\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:27.916168 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.916147    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-maas-api-tls\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:27.916308 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:27.916232    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qst5\" (UniqueName: \"kubernetes.io/projected/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-kube-api-access-4qst5\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.017051 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.016952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-openshift-service-ca\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.017051 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.017045    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-maas-api-tls\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.017239 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.017073    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qst5\" (UniqueName: \"kubernetes.io/projected/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-kube-api-access-4qst5\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.017749 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.017728    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-openshift-service-ca\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.019483 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.019460    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-maas-api-tls\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.037379 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.037358    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qst5\" (UniqueName: \"kubernetes.io/projected/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-kube-api-access-4qst5\") pod \"maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.147542 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.147504    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:28.247506 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.247474    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:28.248613 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.248588    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:28.285807 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.285782    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"]
Jul 02 19:29:28.287943 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:29:28.287915    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4879edeb_56e9_4fea_bbde_2bf7d9f5f367.slice/crio-329f0980a94d5e8881dfa68da4c80e8d617f153b3ac16fbf3be48b9adc257d53 WatchSource:0}: Error finding container 329f0980a94d5e8881dfa68da4c80e8d617f153b3ac16fbf3be48b9adc257d53: Status 404 returned error can't find the container with id 329f0980a94d5e8881dfa68da4c80e8d617f153b3ac16fbf3be48b9adc257d53
Jul 02 19:29:28.289633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:28.289614    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:29:29.253324 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:29.253225    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2" event={"ID":"4879edeb-56e9-4fea-bbde-2bf7d9f5f367","Type":"ContainerStarted","Data":"dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25"}
Jul 02 19:29:29.253324 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:29.253288    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2" event={"ID":"4879edeb-56e9-4fea-bbde-2bf7d9f5f367","Type":"ContainerStarted","Data":"329f0980a94d5e8881dfa68da4c80e8d617f153b3ac16fbf3be48b9adc257d53"}
Jul 02 19:29:29.253816 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:29.253455    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:29.273808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:29.273759    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2" podStartSLOduration=1.618982013 podStartE2EDuration="2.273744887s" totalImagesPullingTime="654.762874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:29:29.272392225 +0000 UTC m=+1549.127323999" watchObservedRunningTime="2026-07-02 19:29:29.273744887 +0000 UTC m=+1549.128676657"
Jul 02 19:29:32.883651 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:32.883612    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"]
Jul 02 19:29:32.884063 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:32.883803    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c" podUID="f16b16bc-aa2d-4ed5-b550-46385ce63b17" containerName="istio-proxy" containerID="cri-o://2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5" gracePeriod=30
Jul 02 19:29:33.012073 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.011534    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"]
Jul 02 19:29:33.012073 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.011763    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2" podUID="4879edeb-56e9-4fea-bbde-2bf7d9f5f367" containerName="maas-api" containerID="cri-o://dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25" gracePeriod=30
Jul 02 19:29:33.019334 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.019309    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:33.253371 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.253332    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:33.272787 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.272750    2584 generic.go:356] "Generic (PLEG): container finished" podID="4879edeb-56e9-4fea-bbde-2bf7d9f5f367" containerID="dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25" exitCode=0
Jul 02 19:29:33.272919 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.272791    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"
Jul 02 19:29:33.272919 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.272831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2" event={"ID":"4879edeb-56e9-4fea-bbde-2bf7d9f5f367","Type":"ContainerDied","Data":"dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25"}
Jul 02 19:29:33.272919 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.272885    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2" event={"ID":"4879edeb-56e9-4fea-bbde-2bf7d9f5f367","Type":"ContainerDied","Data":"329f0980a94d5e8881dfa68da4c80e8d617f153b3ac16fbf3be48b9adc257d53"}
Jul 02 19:29:33.272919 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.272916    2584 scope.go:117] "RemoveContainer" containerID="dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25"
Jul 02 19:29:33.282204 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.282187    2584 scope.go:117] "RemoveContainer" containerID="dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25"
Jul 02 19:29:33.282483 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:33.282461    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25\": container with ID starting with dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25 not found: ID does not exist" containerID="dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25"
Jul 02 19:29:33.282600 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.282488    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25"} err="failed to get container status \"dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25\": rpc error: code = NotFound desc = could not find container \"dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25\": container with ID starting with dea4fff49e6c3ff109bcb651485e0c433ad894de8453017ee1e0e84efb41dc25 not found: ID does not exist"
Jul 02 19:29:33.364913 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.364878    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qst5\" (UniqueName: \"kubernetes.io/projected/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-kube-api-access-4qst5\") pod \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") "
Jul 02 19:29:33.365075 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.364960    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-openshift-service-ca\") pod \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") "
Jul 02 19:29:33.365075 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.365057    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-maas-api-tls\") pod \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\" (UID: \"4879edeb-56e9-4fea-bbde-2bf7d9f5f367\") "
Jul 02 19:29:33.365322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.365301    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4879edeb-56e9-4fea-bbde-2bf7d9f5f367" (UID: "4879edeb-56e9-4fea-bbde-2bf7d9f5f367"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:29:33.367135 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.367109    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4879edeb-56e9-4fea-bbde-2bf7d9f5f367" (UID: "4879edeb-56e9-4fea-bbde-2bf7d9f5f367"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:29:33.367229 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.367208    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-kube-api-access-4qst5" (OuterVolumeSpecName: "kube-api-access-4qst5") pod "4879edeb-56e9-4fea-bbde-2bf7d9f5f367" (UID: "4879edeb-56e9-4fea-bbde-2bf7d9f5f367"). InnerVolumeSpecName "kube-api-access-4qst5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:29:33.466734 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.466653    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:33.466734 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.466679    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:33.466734 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.466690    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qst5\" (UniqueName: \"kubernetes.io/projected/4879edeb-56e9-4fea-bbde-2bf7d9f5f367-kube-api-access-4qst5\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:33.594494 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.594466    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"]
Jul 02 19:29:33.599361 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:33.599309    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3ef64d22-5754669b65-n7sl2"]
Jul 02 19:29:34.776883 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:34.776851    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4879edeb-56e9-4fea-bbde-2bf7d9f5f367" path="/var/lib/kubelet/pods/4879edeb-56e9-4fea-bbde-2bf7d9f5f367/volumes"
Jul 02 19:29:38.120759 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.120735    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:38.296017 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.295925    2584 generic.go:356] "Generic (PLEG): container finished" podID="f16b16bc-aa2d-4ed5-b550-46385ce63b17" containerID="2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5" exitCode=0
Jul 02 19:29:38.296017 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.295977    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"
Jul 02 19:29:38.296017 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.296000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c" event={"ID":"f16b16bc-aa2d-4ed5-b550-46385ce63b17","Type":"ContainerDied","Data":"2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5"}
Jul 02 19:29:38.296216 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.296052    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c" event={"ID":"f16b16bc-aa2d-4ed5-b550-46385ce63b17","Type":"ContainerDied","Data":"7c633136c058d19b49ffbab5998e9d1eb94471195510692378fdcae9fb0dd9ed"}
Jul 02 19:29:38.296216 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.296083    2584 scope.go:117] "RemoveContainer" containerID="2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5"
Jul 02 19:29:38.304222 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.304206    2584 scope.go:117] "RemoveContainer" containerID="2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5"
Jul 02 19:29:38.304490 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:38.304470    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5\": container with ID starting with 2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5 not found: ID does not exist" containerID="2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5"
Jul 02 19:29:38.304547 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.304497    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5"} err="failed to get container status \"2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5\": rpc error: code = NotFound desc = could not find container \"2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5\": container with ID starting with 2a8f386cbe9d9abbec8347e2514725dd257da6fa13e45cae11511a44ef00c8f5 not found: ID does not exist"
Jul 02 19:29:38.306765 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306748    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-podinfo\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.306831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306774    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwjrk\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-kube-api-access-hwjrk\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.306831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306827    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-envoy\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.306937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306879    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-certs\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.306937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306905    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-token\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.306937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306929    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istiod-ca-cert\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.307085 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306952    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-credential-socket\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.307085 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.306986    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-socket\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.307085 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307010    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-data\") pod \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\" (UID: \"f16b16bc-aa2d-4ed5-b550-46385ce63b17\") "
Jul 02 19:29:38.307229 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307171    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:38.307317 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307299    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.307389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307334    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:29:38.307450 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307420    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-data" (OuterVolumeSpecName: "istio-data") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:38.307503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307438    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:38.307563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.307503    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:38.309096 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.309067    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-kube-api-access-hwjrk" (OuterVolumeSpecName: "kube-api-access-hwjrk") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "kube-api-access-hwjrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:29:38.309171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.309103    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:38.309171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.309130    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:29:38.309271 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.309216    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-token" (OuterVolumeSpecName: "istio-token") pod "f16b16bc-aa2d-4ed5-b550-46385ce63b17" (UID: "f16b16bc-aa2d-4ed5-b550-46385ce63b17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:29:38.407767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407728    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.407767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407756    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.407767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407765    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.407767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407773    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.408059 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407782    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.408059 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407791    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.408059 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407798    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f16b16bc-aa2d-4ed5-b550-46385ce63b17-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.408059 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.407806    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwjrk\" (UniqueName: \"kubernetes.io/projected/f16b16bc-aa2d-4ed5-b550-46385ce63b17-kube-api-access-hwjrk\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:38.618111 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.618075    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"]
Jul 02 19:29:38.622308 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.622284    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3ef64d22-openshift-default-75f9b8778c-l6c7c"]
Jul 02 19:29:38.777446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:38.777411    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16b16bc-aa2d-4ed5-b550-46385ce63b17" path="/var/lib/kubelet/pods/f16b16bc-aa2d-4ed5-b550-46385ce63b17/volumes"
Jul 02 19:29:39.388698 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.388660    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"]
Jul 02 19:29:39.389192 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.389173    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16b16bc-aa2d-4ed5-b550-46385ce63b17" containerName="istio-proxy"
Jul 02 19:29:39.389269 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.389195    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16b16bc-aa2d-4ed5-b550-46385ce63b17" containerName="istio-proxy"
Jul 02 19:29:39.389269 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.389207    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4879edeb-56e9-4fea-bbde-2bf7d9f5f367" containerName="maas-api"
Jul 02 19:29:39.389269 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.389215    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4879edeb-56e9-4fea-bbde-2bf7d9f5f367" containerName="maas-api"
Jul 02 19:29:39.389441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.389323    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16b16bc-aa2d-4ed5-b550-46385ce63b17" containerName="istio-proxy"
Jul 02 19:29:39.389441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.389356    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4879edeb-56e9-4fea-bbde-2bf7d9f5f367" containerName="maas-api"
Jul 02 19:29:39.393747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.393723    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.397633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.397604    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ce0fe37f-openshift-default-dockercfg-xz2v5\""
Jul 02 19:29:39.405929 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.405902    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"]
Jul 02 19:29:39.516284 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516245    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-certs\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516583 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516563    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6vmh\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-kube-api-access-z6vmh\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516755 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516712    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-token\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516762    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-envoy\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516787    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-data\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516832    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-socket\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istiod-ca-cert\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.516912 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-podinfo\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.517255 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.516924    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-credential-socket\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618403 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618336    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-certs\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618569 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618421    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6vmh\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-kube-api-access-z6vmh\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618569 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-token\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618569 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-envoy\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618674 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618623    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-data\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618708 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618688    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-socket\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618745 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istiod-ca-cert\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618794 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-podinfo\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618794 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618786    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-certs\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.618906 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-credential-socket\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.619010 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.618986    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-data\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.619099 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.619075    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-socket\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.619190 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.619163    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-credential-socket\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.619444 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.619426    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istiod-ca-cert\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.620735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.620708    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-envoy\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.621009 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.620992    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-podinfo\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.626967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.626940    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-token\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.627055 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.627030    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6vmh\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-kube-api-access-z6vmh\") pod \"e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") " pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.707097 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.706998    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:39.842398 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.842371    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"]
Jul 02 19:29:39.844686 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:29:39.844648    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df2257a_c2a0_42c1_bddd_7d951e60d90a.slice/crio-ae28b5d859e1259969c58b5b985bc494c2601e7dbfbbffdb36887f27f579772c WatchSource:0}: Error finding container ae28b5d859e1259969c58b5b985bc494c2601e7dbfbbffdb36887f27f579772c: Status 404 returned error can't find the container with id ae28b5d859e1259969c58b5b985bc494c2601e7dbfbbffdb36887f27f579772c
Jul 02 19:29:39.846942 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.846901    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:39.847038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.846991    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:39.847038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:39.847032    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:40.307169 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.307133    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" event={"ID":"2df2257a-c2a0-42c1-bddd-7d951e60d90a","Type":"ContainerStarted","Data":"32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"}
Jul 02 19:29:40.307364 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.307184    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" event={"ID":"2df2257a-c2a0-42c1-bddd-7d951e60d90a","Type":"ContainerStarted","Data":"ae28b5d859e1259969c58b5b985bc494c2601e7dbfbbffdb36887f27f579772c"}
Jul 02 19:29:40.348181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.348132    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" podStartSLOduration=1.3481159489999999 podStartE2EDuration="1.348115949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:29:40.346491214 +0000 UTC m=+1560.201422984" watchObservedRunningTime="2026-07-02 19:29:40.348115949 +0000 UTC m=+1560.203047718"
Jul 02 19:29:40.700936 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.700903    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"]
Jul 02 19:29:40.704274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.704250    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.706744 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.706712    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ce0fe37f-serving-cert\""
Jul 02 19:29:40.707332 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.707314    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:40.714160 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.714138    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:40.715768 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.715694    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"]
Jul 02 19:29:40.830833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.830797    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsf7s\" (UniqueName: \"kubernetes.io/projected/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-kube-api-access-nsf7s\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.831039 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.831010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.831103 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.831063    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-openshift-service-ca\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.932038 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.932011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.932197 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.932043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-openshift-service-ca\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.932197 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.932090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsf7s\" (UniqueName: \"kubernetes.io/projected/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-kube-api-access-nsf7s\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.932197 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:40.932180    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-serving-cert: secret "maas-api-e2e-ait-ce0fe37f-serving-cert" not found
Jul 02 19:29:40.932381 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:40.932258    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls podName:ece1064e-66e5-48ea-a4b3-47e79b69dc8c nodeName:}" failed. No retries permitted until 2026-07-02 19:29:41.432234001 +0000 UTC m=+1561.287165751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls") pod "maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" (UID: "ece1064e-66e5-48ea-a4b3-47e79b69dc8c") : secret "maas-api-e2e-ait-ce0fe37f-serving-cert" not found
Jul 02 19:29:40.932731 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.932712    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-openshift-service-ca\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:40.949466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:40.949443    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsf7s\" (UniqueName: \"kubernetes.io/projected/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-kube-api-access-nsf7s\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:41.311397 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:41.311370    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:41.312317 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:41.312295    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:41.438397 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:41.438368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:41.440740 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:41.440699    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls\") pod \"maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:41.620268 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:41.620237    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:41.753930 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:41.753904    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"]
Jul 02 19:29:41.756501 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:29:41.756473    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece1064e_66e5_48ea_a4b3_47e79b69dc8c.slice/crio-42f9d46cedb3e7f29304f948e2a0a799f01e0d16e3fe6099d6bc50818feea840 WatchSource:0}: Error finding container 42f9d46cedb3e7f29304f948e2a0a799f01e0d16e3fe6099d6bc50818feea840: Status 404 returned error can't find the container with id 42f9d46cedb3e7f29304f948e2a0a799f01e0d16e3fe6099d6bc50818feea840
Jul 02 19:29:42.317049 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:42.317012    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" event={"ID":"ece1064e-66e5-48ea-a4b3-47e79b69dc8c","Type":"ContainerStarted","Data":"b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364"}
Jul 02 19:29:42.317200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:42.317111    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:42.317200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:42.317135    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" event={"ID":"ece1064e-66e5-48ea-a4b3-47e79b69dc8c","Type":"ContainerStarted","Data":"42f9d46cedb3e7f29304f948e2a0a799f01e0d16e3fe6099d6bc50818feea840"}
Jul 02 19:29:42.337289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:42.337241    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" podStartSLOduration=1.865931668 podStartE2EDuration="2.33722607s" totalImagesPullingTime="471.294402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:29:42.335706002 +0000 UTC m=+1562.190637773" watchObservedRunningTime="2026-07-02 19:29:42.33722607 +0000 UTC m=+1562.192157840"
Jul 02 19:29:46.304416 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.304385    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"]
Jul 02 19:29:46.304788 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.304567    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" podUID="ece1064e-66e5-48ea-a4b3-47e79b69dc8c" containerName="maas-api" containerID="cri-o://b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364" gracePeriod=30
Jul 02 19:29:46.578786 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.578756    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:46.683158 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.683117    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-openshift-service-ca\") pod \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") "
Jul 02 19:29:46.683158 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.683162    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsf7s\" (UniqueName: \"kubernetes.io/projected/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-kube-api-access-nsf7s\") pod \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") "
Jul 02 19:29:46.683434 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.683203    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls\") pod \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\" (UID: \"ece1064e-66e5-48ea-a4b3-47e79b69dc8c\") "
Jul 02 19:29:46.683519 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.683495    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ece1064e-66e5-48ea-a4b3-47e79b69dc8c" (UID: "ece1064e-66e5-48ea-a4b3-47e79b69dc8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:29:46.685285 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.685259    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ece1064e-66e5-48ea-a4b3-47e79b69dc8c" (UID: "ece1064e-66e5-48ea-a4b3-47e79b69dc8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:29:46.685466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.685443    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-kube-api-access-nsf7s" (OuterVolumeSpecName: "kube-api-access-nsf7s") pod "ece1064e-66e5-48ea-a4b3-47e79b69dc8c" (UID: "ece1064e-66e5-48ea-a4b3-47e79b69dc8c"). InnerVolumeSpecName "kube-api-access-nsf7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:29:46.784022 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.783993    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:46.784105 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.784023    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsf7s\" (UniqueName: \"kubernetes.io/projected/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-kube-api-access-nsf7s\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:46.784105 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:46.784037    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ece1064e-66e5-48ea-a4b3-47e79b69dc8c-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:47.359227 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.359193    2584 generic.go:356] "Generic (PLEG): container finished" podID="ece1064e-66e5-48ea-a4b3-47e79b69dc8c" containerID="b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364" exitCode=0
Jul 02 19:29:47.359747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.359242    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"
Jul 02 19:29:47.359747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.359278    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" event={"ID":"ece1064e-66e5-48ea-a4b3-47e79b69dc8c","Type":"ContainerDied","Data":"b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364"}
Jul 02 19:29:47.359747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.359360    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h" event={"ID":"ece1064e-66e5-48ea-a4b3-47e79b69dc8c","Type":"ContainerDied","Data":"42f9d46cedb3e7f29304f948e2a0a799f01e0d16e3fe6099d6bc50818feea840"}
Jul 02 19:29:47.359747 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.359393    2584 scope.go:117] "RemoveContainer" containerID="b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364"
Jul 02 19:29:47.367725 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.367704    2584 scope.go:117] "RemoveContainer" containerID="b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364"
Jul 02 19:29:47.367982 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:47.367964    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364\": container with ID starting with b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364 not found: ID does not exist" containerID="b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364"
Jul 02 19:29:47.368044 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.367988    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364"} err="failed to get container status \"b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364\": rpc error: code = NotFound desc = could not find container \"b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364\": container with ID starting with b21e2e03dce7703fd657cbfe26add83035a935b5f42f1dbaa23b9471dff44364 not found: ID does not exist"
Jul 02 19:29:47.376356 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.376318    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"]
Jul 02 19:29:47.379053 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:47.379034    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce0fe37f-6bdddf759b-xbj8h"]
Jul 02 19:29:48.777017 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:48.776979    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece1064e-66e5-48ea-a4b3-47e79b69dc8c" path="/var/lib/kubelet/pods/ece1064e-66e5-48ea-a4b3-47e79b69dc8c/volumes"
Jul 02 19:29:51.378165 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:51.378131    2584 generic.go:356] "Generic (PLEG): container finished" podID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerID="32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9" exitCode=0
Jul 02 19:29:51.378646 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:51.378195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" event={"ID":"2df2257a-c2a0-42c1-bddd-7d951e60d90a","Type":"ContainerDied","Data":"32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"}
Jul 02 19:29:51.378646 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:51.378579    2584 scope.go:117] "RemoveContainer" containerID="32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"
Jul 02 19:29:51.379543 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:51.379513    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:51.379611 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:51.379584    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:51.379647 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:51.379627    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:52.151806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:52.151767    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"]
Jul 02 19:29:52.384392 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:52.384331    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" event={"ID":"2df2257a-c2a0-42c1-bddd-7d951e60d90a","Type":"ContainerStarted","Data":"92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563"}
Jul 02 19:29:52.384788 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:52.384449    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy" containerID="cri-o://92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563" gracePeriod=30
Jul 02 19:29:56.311544 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:56.311510    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:57.639415 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.639387    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:57.678976 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.678886    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-credential-socket\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.678976 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.678944    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-podinfo\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.678984    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-certs\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679012    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6vmh\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-kube-api-access-z6vmh\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679119    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:57.679203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679140    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-token\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679203    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-socket\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679216    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:57.679446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679248    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-envoy\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679283    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-data\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679315    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istiod-ca-cert\") pod \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\" (UID: \"2df2257a-c2a0-42c1-bddd-7d951e60d90a\") "
Jul 02 19:29:57.679446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679442    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:57.679731 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679674    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.679731 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679694    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.679731 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679709    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.679885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.679782    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:29:57.680634 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.680605    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:57.681296 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.681268    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-kube-api-access-z6vmh" (OuterVolumeSpecName: "kube-api-access-z6vmh") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "kube-api-access-z6vmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:29:57.681465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.681443    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:29:57.681543 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.681450    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:29:57.681620 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.681603    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2df2257a-c2a0-42c1-bddd-7d951e60d90a" (UID: "2df2257a-c2a0-42c1-bddd-7d951e60d90a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:29:57.780905 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.780869    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.780905 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.780902    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6vmh\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-kube-api-access-z6vmh\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.781100 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.780917    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.781100 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.780930    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.781100 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.780941    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:57.781100 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:57.780953    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2df2257a-c2a0-42c1-bddd-7d951e60d90a-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:29:58.412648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.412609    2584 generic.go:356] "Generic (PLEG): container finished" podID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerID="92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563" exitCode=0
Jul 02 19:29:58.412838 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.412660    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"
Jul 02 19:29:58.412838 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.412695    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" event={"ID":"2df2257a-c2a0-42c1-bddd-7d951e60d90a","Type":"ContainerDied","Data":"92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563"}
Jul 02 19:29:58.412838 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.412751    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r" event={"ID":"2df2257a-c2a0-42c1-bddd-7d951e60d90a","Type":"ContainerDied","Data":"ae28b5d859e1259969c58b5b985bc494c2601e7dbfbbffdb36887f27f579772c"}
Jul 02 19:29:58.412838 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.412773    2584 scope.go:117] "RemoveContainer" containerID="92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563"
Jul 02 19:29:58.421602 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.421583    2584 scope.go:117] "RemoveContainer" containerID="32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"
Jul 02 19:29:58.429135 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.429115    2584 scope.go:117] "RemoveContainer" containerID="92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563"
Jul 02 19:29:58.429411 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:58.429393    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563\": container with ID starting with 92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563 not found: ID does not exist" containerID="92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563"
Jul 02 19:29:58.429461 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.429420    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563"} err="failed to get container status \"92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563\": rpc error: code = NotFound desc = could not find container \"92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563\": container with ID starting with 92d94c6187c6da59795b0037d0cae6099cdd970409d19d381a091fa2bdb15563 not found: ID does not exist"
Jul 02 19:29:58.429461 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.429437    2584 scope.go:117] "RemoveContainer" containerID="32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"
Jul 02 19:29:58.429656 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:29:58.429638    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9\": container with ID starting with 32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9 not found: ID does not exist" containerID="32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"
Jul 02 19:29:58.429697 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.429661    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9"} err="failed to get container status \"32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9\": rpc error: code = NotFound desc = could not find container \"32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9\": container with ID starting with 32f4c977f05e6ee472d3f47d4f0acb9670626dc08490c7ae30a340acc44ee7e9 not found: ID does not exist"
Jul 02 19:29:58.443080 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.443053    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"]
Jul 02 19:29:58.447280 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447254    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"]
Jul 02 19:29:58.447758 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447733    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy"
Jul 02 19:29:58.447758 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447757    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy"
Jul 02 19:29:58.447938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447782    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece1064e-66e5-48ea-a4b3-47e79b69dc8c" containerName="maas-api"
Jul 02 19:29:58.447938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447791    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece1064e-66e5-48ea-a4b3-47e79b69dc8c" containerName="maas-api"
Jul 02 19:29:58.447938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447878    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece1064e-66e5-48ea-a4b3-47e79b69dc8c" containerName="maas-api"
Jul 02 19:29:58.447938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447888    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy"
Jul 02 19:29:58.447938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.447898    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy"
Jul 02 19:29:58.452798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.452779    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.455903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.455853    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-de3868c8-openshift-default-dockercfg-jvmfv\""
Jul 02 19:29:58.457997 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.457979    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ce0fe37f-openshift-default-7959c75949-mn76r"]
Jul 02 19:29:58.462923 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.462903    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"]
Jul 02 19:29:58.486146 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-envoy\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486250 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486185    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-data\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486250 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486238    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-certs\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486325 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-socket\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486325 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-token\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486392    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-credential-socket\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486412    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istiod-ca-cert\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486430    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wthcw\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-kube-api-access-wthcw\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.486505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.486475    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-podinfo\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587326 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-socket\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587326 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587332    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-token\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-credential-socket\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istiod-ca-cert\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587426    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wthcw\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-kube-api-access-wthcw\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587483    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-podinfo\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587729 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-envoy\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587814 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587742    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-socket\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587814 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587748    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-data\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587814 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587776    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-credential-socket\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.587971 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.587862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-certs\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.588037 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.588014    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-data\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.588139 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.588117    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-certs\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.588259 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.588208    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istiod-ca-cert\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.589740 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.589716    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-envoy\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.590000 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.589981    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-podinfo\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.596011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.595991    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-token\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.596158 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.596137    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wthcw\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-kube-api-access-wthcw\") pod \"e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") " pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.764196 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.764111    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:58.777915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.777882    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" path="/var/lib/kubelet/pods/2df2257a-c2a0-42c1-bddd-7d951e60d90a/volumes"
Jul 02 19:29:58.905858 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.901910    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"]
Jul 02 19:29:58.910539 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:29:58.910487    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6981c4ce_54ea_4ba8_9a7c_6461148211d0.slice/crio-a68139f6430b4769b319a0ab48289c8c9ff022556038eea441f81fbb562fe695 WatchSource:0}: Error finding container a68139f6430b4769b319a0ab48289c8c9ff022556038eea441f81fbb562fe695: Status 404 returned error can't find the container with id a68139f6430b4769b319a0ab48289c8c9ff022556038eea441f81fbb562fe695
Jul 02 19:29:58.913022 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.912975    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:58.913136 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.913064    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:58.913136 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:58.913105    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:29:59.418642 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.418603    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p" event={"ID":"6981c4ce-54ea-4ba8-9a7c-6461148211d0","Type":"ContainerStarted","Data":"bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2"}
Jul 02 19:29:59.418830 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.418666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p" event={"ID":"6981c4ce-54ea-4ba8-9a7c-6461148211d0","Type":"ContainerStarted","Data":"a68139f6430b4769b319a0ab48289c8c9ff022556038eea441f81fbb562fe695"}
Jul 02 19:29:59.443465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.443415    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p" podStartSLOduration=1.4433982269999999 podStartE2EDuration="1.443398227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:29:59.441818441 +0000 UTC m=+1579.296750208" watchObservedRunningTime="2026-07-02 19:29:59.443398227 +0000 UTC m=+1579.298329997"
Jul 02 19:29:59.764415 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.764311    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:59.769155 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.769132    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:29:59.921508 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.921480    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"]
Jul 02 19:29:59.921808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.921796    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy"
Jul 02 19:29:59.921874 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.921810    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df2257a-c2a0-42c1-bddd-7d951e60d90a" containerName="istio-proxy"
Jul 02 19:29:59.924819 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.924798    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:29:59.927399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.927376    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-de3868c8-serving-cert\""
Jul 02 19:29:59.933797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:29:59.933775    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"]
Jul 02 19:30:00.000552 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.000508    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de5f1d24-7fd0-42c3-9076-448b70bb07f6-openshift-service-ca\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.000732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.000614    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.000732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.000644    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czwgh\" (UniqueName: \"kubernetes.io/projected/de5f1d24-7fd0-42c3-9076-448b70bb07f6-kube-api-access-czwgh\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.101871 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.101840    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de5f1d24-7fd0-42c3-9076-448b70bb07f6-openshift-service-ca\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.102068 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.101941    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.102068 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.101968    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czwgh\" (UniqueName: \"kubernetes.io/projected/de5f1d24-7fd0-42c3-9076-448b70bb07f6-kube-api-access-czwgh\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.102211 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:30:00.102091    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-serving-cert: secret "maas-api-e2e-derive-de3868c8-serving-cert" not found
Jul 02 19:30:00.102211 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:30:00.102160    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls podName:de5f1d24-7fd0-42c3-9076-448b70bb07f6 nodeName:}" failed. No retries permitted until 2026-07-02 19:30:00.60214344 +0000 UTC m=+1580.457075201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls") pod "maas-api-e2e-derive-de3868c8-6989866dff-c46gz" (UID: "de5f1d24-7fd0-42c3-9076-448b70bb07f6") : secret "maas-api-e2e-derive-de3868c8-serving-cert" not found
Jul 02 19:30:00.102549 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.102530    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de5f1d24-7fd0-42c3-9076-448b70bb07f6-openshift-service-ca\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.117497 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.117478    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czwgh\" (UniqueName: \"kubernetes.io/projected/de5f1d24-7fd0-42c3-9076-448b70bb07f6-kube-api-access-czwgh\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.424567 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.424475    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:30:00.425511 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.425491    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:30:00.606359 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:00.606312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:00.606523 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:30:00.606454    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-serving-cert: secret "maas-api-e2e-derive-de3868c8-serving-cert" not found
Jul 02 19:30:00.606523 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:30:00.606521    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls podName:de5f1d24-7fd0-42c3-9076-448b70bb07f6 nodeName:}" failed. No retries permitted until 2026-07-02 19:30:01.606501525 +0000 UTC m=+1581.461433275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls") pod "maas-api-e2e-derive-de3868c8-6989866dff-c46gz" (UID: "de5f1d24-7fd0-42c3-9076-448b70bb07f6") : secret "maas-api-e2e-derive-de3868c8-serving-cert" not found
Jul 02 19:30:01.617539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:01.617496    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:01.619975 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:01.619956    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") pod \"maas-api-e2e-derive-de3868c8-6989866dff-c46gz\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:01.737253 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:01.737221    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:01.899527 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:01.899498    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"]
Jul 02 19:30:01.900992 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:30:01.900957    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5f1d24_7fd0_42c3_9076_448b70bb07f6.slice/crio-7c39b50534f144e41c4fc6bc0eabda1d4e0d68019d7b4a5c85fbf11c190916b5 WatchSource:0}: Error finding container 7c39b50534f144e41c4fc6bc0eabda1d4e0d68019d7b4a5c85fbf11c190916b5: Status 404 returned error can't find the container with id 7c39b50534f144e41c4fc6bc0eabda1d4e0d68019d7b4a5c85fbf11c190916b5
Jul 02 19:30:02.435642 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:02.435610    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz" event={"ID":"de5f1d24-7fd0-42c3-9076-448b70bb07f6","Type":"ContainerStarted","Data":"3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657"}
Jul 02 19:30:02.435808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:02.435656    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz" event={"ID":"de5f1d24-7fd0-42c3-9076-448b70bb07f6","Type":"ContainerStarted","Data":"7c39b50534f144e41c4fc6bc0eabda1d4e0d68019d7b4a5c85fbf11c190916b5"}
Jul 02 19:30:02.456931 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:02.456886    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz" podStartSLOduration=3.032521638 podStartE2EDuration="3.456869947s" totalImagesPullingTime="424.348309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:30:02.45559886 +0000 UTC m=+1582.310530633" watchObservedRunningTime="2026-07-02 19:30:02.456869947 +0000 UTC m=+1582.311801716"
Jul 02 19:30:03.440125 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:03.440095    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:04.342300 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:04.342251    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"]
Jul 02 19:30:04.343399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:04.343047    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p" podUID="6981c4ce-54ea-4ba8-9a7c-6461148211d0" containerName="istio-proxy" containerID="cri-o://bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2" gracePeriod=30
Jul 02 19:30:04.610683 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:04.610601    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"]
Jul 02 19:30:05.453735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.453688    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz" podUID="de5f1d24-7fd0-42c3-9076-448b70bb07f6" containerName="maas-api" containerID="cri-o://3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657" gracePeriod=30
Jul 02 19:30:05.711055 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.711001    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:05.760883 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.760846    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") pod \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") "
Jul 02 19:30:05.761039 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.760941    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de5f1d24-7fd0-42c3-9076-448b70bb07f6-openshift-service-ca\") pod \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") "
Jul 02 19:30:05.761039 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.760997    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czwgh\" (UniqueName: \"kubernetes.io/projected/de5f1d24-7fd0-42c3-9076-448b70bb07f6-kube-api-access-czwgh\") pod \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\" (UID: \"de5f1d24-7fd0-42c3-9076-448b70bb07f6\") "
Jul 02 19:30:05.761269 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.761242    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de5f1d24-7fd0-42c3-9076-448b70bb07f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de5f1d24-7fd0-42c3-9076-448b70bb07f6" (UID: "de5f1d24-7fd0-42c3-9076-448b70bb07f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:30:05.762953 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.762924    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de5f1d24-7fd0-42c3-9076-448b70bb07f6" (UID: "de5f1d24-7fd0-42c3-9076-448b70bb07f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:30:05.763085 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.763025    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5f1d24-7fd0-42c3-9076-448b70bb07f6-kube-api-access-czwgh" (OuterVolumeSpecName: "kube-api-access-czwgh") pod "de5f1d24-7fd0-42c3-9076-448b70bb07f6" (UID: "de5f1d24-7fd0-42c3-9076-448b70bb07f6"). InnerVolumeSpecName "kube-api-access-czwgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:30:05.862635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.862603    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de5f1d24-7fd0-42c3-9076-448b70bb07f6-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:05.862635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.862629    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de5f1d24-7fd0-42c3-9076-448b70bb07f6-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:05.862635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:05.862638    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czwgh\" (UniqueName: \"kubernetes.io/projected/de5f1d24-7fd0-42c3-9076-448b70bb07f6-kube-api-access-czwgh\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:06.459569 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.459533    2584 generic.go:356] "Generic (PLEG): container finished" podID="de5f1d24-7fd0-42c3-9076-448b70bb07f6" containerID="3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657" exitCode=0
Jul 02 19:30:06.459739 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.459578    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"
Jul 02 19:30:06.459739 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.459613    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz" event={"ID":"de5f1d24-7fd0-42c3-9076-448b70bb07f6","Type":"ContainerDied","Data":"3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657"}
Jul 02 19:30:06.459739 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.459659    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz" event={"ID":"de5f1d24-7fd0-42c3-9076-448b70bb07f6","Type":"ContainerDied","Data":"7c39b50534f144e41c4fc6bc0eabda1d4e0d68019d7b4a5c85fbf11c190916b5"}
Jul 02 19:30:06.459739 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.459685    2584 scope.go:117] "RemoveContainer" containerID="3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657"
Jul 02 19:30:06.469073 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.469055    2584 scope.go:117] "RemoveContainer" containerID="3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657"
Jul 02 19:30:06.469350 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:30:06.469313    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657\": container with ID starting with 3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657 not found: ID does not exist" containerID="3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657"
Jul 02 19:30:06.469425 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.469357    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657"} err="failed to get container status \"3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657\": rpc error: code = NotFound desc = could not find container \"3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657\": container with ID starting with 3d1196debe6ca1e8374ed7296610764fa2755b3b7714c41ed5b8a04e65ec8657 not found: ID does not exist"
Jul 02 19:30:06.481918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.481892    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"]
Jul 02 19:30:06.485751 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.485729    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-de3868c8-6989866dff-c46gz"]
Jul 02 19:30:06.776604 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:06.776517    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de5f1d24-7fd0-42c3-9076-448b70bb07f6" path="/var/lib/kubelet/pods/de5f1d24-7fd0-42c3-9076-448b70bb07f6/volumes"
Jul 02 19:30:11.008120 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.008093    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:30:11.106609 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106582    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-certs\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.106779 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106628    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istiod-ca-cert\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.106779 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106663    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-data\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.106779 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106692    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-token\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.106779 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106713    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-envoy\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.106779 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106734    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-socket\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.107047 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106803    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-podinfo\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.107047 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106833    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:30:11.107047 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106869    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wthcw\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-kube-api-access-wthcw\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.107047 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.106899    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-credential-socket\") pod \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\" (UID: \"6981c4ce-54ea-4ba8-9a7c-6461148211d0\") "
Jul 02 19:30:11.107256 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.107161    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.107256 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.107198    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:30:11.107256 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.107199    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:30:11.107505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.107335    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:30:11.107950 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.107923    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:30:11.108970 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.108944    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:30:11.109061 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.109023    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:30:11.109061 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.109036    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:30:11.109130 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.109082    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-kube-api-access-wthcw" (OuterVolumeSpecName: "kube-api-access-wthcw") pod "6981c4ce-54ea-4ba8-9a7c-6461148211d0" (UID: "6981c4ce-54ea-4ba8-9a7c-6461148211d0"). InnerVolumeSpecName "kube-api-access-wthcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:30:11.207581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207548    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207578    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wthcw\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-kube-api-access-wthcw\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207593    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207606    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207617    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207627    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207638    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.207766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.207649    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6981c4ce-54ea-4ba8-9a7c-6461148211d0-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:30:11.483716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.483624    2584 generic.go:356] "Generic (PLEG): container finished" podID="6981c4ce-54ea-4ba8-9a7c-6461148211d0" containerID="bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2" exitCode=0
Jul 02 19:30:11.483716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.483667    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p" event={"ID":"6981c4ce-54ea-4ba8-9a7c-6461148211d0","Type":"ContainerDied","Data":"bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2"}
Jul 02 19:30:11.483716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.483674    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"
Jul 02 19:30:11.483716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.483708    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p" event={"ID":"6981c4ce-54ea-4ba8-9a7c-6461148211d0","Type":"ContainerDied","Data":"a68139f6430b4769b319a0ab48289c8c9ff022556038eea441f81fbb562fe695"}
Jul 02 19:30:11.484044 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.483729    2584 scope.go:117] "RemoveContainer" containerID="bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2"
Jul 02 19:30:11.492889 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.492869    2584 scope.go:117] "RemoveContainer" containerID="bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2"
Jul 02 19:30:11.493110 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:30:11.493093    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2\": container with ID starting with bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2 not found: ID does not exist" containerID="bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2"
Jul 02 19:30:11.493157 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.493119    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2"} err="failed to get container status \"bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2\": rpc error: code = NotFound desc = could not find container \"bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2\": container with ID starting with bfc6bbb9cd9de2705a2114bf99bbf4313d1843da6179441e7ff7f5d8cd4871d2 not found: ID does not exist"
Jul 02 19:30:11.522558 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.522532    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"]
Jul 02 19:30:11.528868 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:11.528846    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-de3868c8-openshift-default-776b5d578c-d7b4p"]
Jul 02 19:30:12.777200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:30:12.777165    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6981c4ce-54ea-4ba8-9a7c-6461148211d0" path="/var/lib/kubelet/pods/6981c4ce-54ea-4ba8-9a7c-6461148211d0/volumes"
Jul 02 19:31:46.471993 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.471957    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"]
Jul 02 19:31:46.472443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.472285    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6981c4ce-54ea-4ba8-9a7c-6461148211d0" containerName="istio-proxy"
Jul 02 19:31:46.472443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.472296    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6981c4ce-54ea-4ba8-9a7c-6461148211d0" containerName="istio-proxy"
Jul 02 19:31:46.472443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.472313    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5f1d24-7fd0-42c3-9076-448b70bb07f6" containerName="maas-api"
Jul 02 19:31:46.472443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.472319    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5f1d24-7fd0-42c3-9076-448b70bb07f6" containerName="maas-api"
Jul 02 19:31:46.472443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.472387    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6981c4ce-54ea-4ba8-9a7c-6461148211d0" containerName="istio-proxy"
Jul 02 19:31:46.472443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.472399    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5f1d24-7fd0-42c3-9076-448b70bb07f6" containerName="maas-api"
Jul 02 19:31:46.475151 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.475135    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.478533 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.478508    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-47a0d36c-openshift-default-dockercfg-7hs9l\""
Jul 02 19:31:46.501361 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.501313    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"]
Jul 02 19:31:46.528158 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-data\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528167    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-certs\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528196    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-token\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528225    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67ctv\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-kube-api-access-67ctv\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528271    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-credential-socket\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528473 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-socket\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528473 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528402    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-podinfo\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528473 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528440    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istiod-ca-cert\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.528579 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.528483    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-envoy\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-podinfo\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istiod-ca-cert\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629805 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629616    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-envoy\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629805 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629664    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-data\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629805 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629692    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-certs\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629805 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629730    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-token\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.629805 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629753    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67ctv\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-kube-api-access-67ctv\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630021 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629822    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-credential-socket\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630021 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.629850    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-socket\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630183 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.630155    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-certs\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.630253    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-data\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630404 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.630377    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-credential-socket\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630404 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.630394    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istiod-ca-cert\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.630560 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.630468    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-socket\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.632011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.631988    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-envoy\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.632136 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.632121    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-podinfo\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.638259 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.638238    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-token\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.638487 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.638465    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67ctv\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-kube-api-access-67ctv\") pod \"e2e-mt-47a0d36c-openshift-default-666687647-p4j94\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") " pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.785264 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.785184    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:46.948664 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.948638    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"]
Jul 02 19:31:46.950444 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:31:46.950411    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e57796a_9109_41a5_9c3b_f74a7934b6f1.slice/crio-47cec0868081f3fe14f2f52468218447ded35c3483305a8021ff3dcd203b13d4 WatchSource:0}: Error finding container 47cec0868081f3fe14f2f52468218447ded35c3483305a8021ff3dcd203b13d4: Status 404 returned error can't find the container with id 47cec0868081f3fe14f2f52468218447ded35c3483305a8021ff3dcd203b13d4
Jul 02 19:31:46.952485 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.952449    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:31:46.952587 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.952532    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:31:46.952587 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:46.952575    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:31:47.910198 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:47.910164    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" event={"ID":"9e57796a-9109-41a5-9c3b-f74a7934b6f1","Type":"ContainerStarted","Data":"e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"}
Jul 02 19:31:47.910579 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:47.910222    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" event={"ID":"9e57796a-9109-41a5-9c3b-f74a7934b6f1","Type":"ContainerStarted","Data":"47cec0868081f3fe14f2f52468218447ded35c3483305a8021ff3dcd203b13d4"}
Jul 02 19:31:47.935291 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:47.935241    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" podStartSLOduration=1.9352257370000001 podStartE2EDuration="1.935225737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:31:47.933595728 +0000 UTC m=+1687.788527500" watchObservedRunningTime="2026-07-02 19:31:47.935225737 +0000 UTC m=+1687.790157507"
Jul 02 19:31:48.786202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:48.786177    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:48.791013 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:48.790993    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:48.914877 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:48.914844    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:48.915991 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:48.915972    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:31:50.030706 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.030670    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"]
Jul 02 19:31:50.034556 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.034532    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.036872 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.036851    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-47a0d36c-serving-cert\""
Jul 02 19:31:50.043556 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.043532    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"]
Jul 02 19:31:50.058004 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.057979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ce37b6-bdca-4a13-8111-3c2a08174e9c-openshift-service-ca\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.058138 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.058011    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpw49\" (UniqueName: \"kubernetes.io/projected/44ce37b6-bdca-4a13-8111-3c2a08174e9c-kube-api-access-hpw49\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.058202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.058168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.159373 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.159325    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ce37b6-bdca-4a13-8111-3c2a08174e9c-openshift-service-ca\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.159581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.159383    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpw49\" (UniqueName: \"kubernetes.io/projected/44ce37b6-bdca-4a13-8111-3c2a08174e9c-kube-api-access-hpw49\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.159581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.159489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.159704 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:31:50.159608    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-serving-cert: secret "maas-api-e2e-mt-47a0d36c-serving-cert" not found
Jul 02 19:31:50.159704 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:31:50.159673    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls podName:44ce37b6-bdca-4a13-8111-3c2a08174e9c nodeName:}" failed. No retries permitted until 2026-07-02 19:31:50.659649775 +0000 UTC m=+1690.514581527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls") pod "maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" (UID: "44ce37b6-bdca-4a13-8111-3c2a08174e9c") : secret "maas-api-e2e-mt-47a0d36c-serving-cert" not found
Jul 02 19:31:50.160072 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.160051    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ce37b6-bdca-4a13-8111-3c2a08174e9c-openshift-service-ca\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.175110 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.175083    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpw49\" (UniqueName: \"kubernetes.io/projected/44ce37b6-bdca-4a13-8111-3c2a08174e9c-kube-api-access-hpw49\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.663787 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.663754    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.666276 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.666250    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls\") pod \"maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:50.947551 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:50.947465    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:51.083120 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:51.083098    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"]
Jul 02 19:31:51.085185 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:31:51.085159    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ce37b6_bdca_4a13_8111_3c2a08174e9c.slice/crio-c5b3e92529dfe0c7a5a988f12c11c529c1eb8bd52ef0da640cfc16f68b43a1fd WatchSource:0}: Error finding container c5b3e92529dfe0c7a5a988f12c11c529c1eb8bd52ef0da640cfc16f68b43a1fd: Status 404 returned error can't find the container with id c5b3e92529dfe0c7a5a988f12c11c529c1eb8bd52ef0da640cfc16f68b43a1fd
Jul 02 19:31:51.929519 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:51.929484    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" event={"ID":"44ce37b6-bdca-4a13-8111-3c2a08174e9c","Type":"ContainerStarted","Data":"330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c"}
Jul 02 19:31:51.929687 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:51.929536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" event={"ID":"44ce37b6-bdca-4a13-8111-3c2a08174e9c","Type":"ContainerStarted","Data":"c5b3e92529dfe0c7a5a988f12c11c529c1eb8bd52ef0da640cfc16f68b43a1fd"}
Jul 02 19:31:51.929687 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:51.929566    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:31:51.950706 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:51.950659    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" podStartSLOduration=1.5275047910000001 podStartE2EDuration="1.95064305s" totalImagesPullingTime="423.138259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:31:51.94917338 +0000 UTC m=+1691.804105146" watchObservedRunningTime="2026-07-02 19:31:51.95064305 +0000 UTC m=+1691.805574820"
Jul 02 19:31:57.951686 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:31:57.951648    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:32:02.198018 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.197938    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"]
Jul 02 19:32:02.198435 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.198219    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" podUID="44ce37b6-bdca-4a13-8111-3c2a08174e9c" containerName="maas-api" containerID="cri-o://330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c" gracePeriod=30
Jul 02 19:32:02.455291 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.455238    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:32:02.579962 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.579926    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls\") pod \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") "
Jul 02 19:32:02.580147 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.580071    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ce37b6-bdca-4a13-8111-3c2a08174e9c-openshift-service-ca\") pod \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") "
Jul 02 19:32:02.580147 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.580104    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpw49\" (UniqueName: \"kubernetes.io/projected/44ce37b6-bdca-4a13-8111-3c2a08174e9c-kube-api-access-hpw49\") pod \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\" (UID: \"44ce37b6-bdca-4a13-8111-3c2a08174e9c\") "
Jul 02 19:32:02.580440 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.580418    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44ce37b6-bdca-4a13-8111-3c2a08174e9c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44ce37b6-bdca-4a13-8111-3c2a08174e9c" (UID: "44ce37b6-bdca-4a13-8111-3c2a08174e9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:32:02.582098 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.582060    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44ce37b6-bdca-4a13-8111-3c2a08174e9c" (UID: "44ce37b6-bdca-4a13-8111-3c2a08174e9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:32:02.582220 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.582191    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44ce37b6-bdca-4a13-8111-3c2a08174e9c-kube-api-access-hpw49" (OuterVolumeSpecName: "kube-api-access-hpw49") pod "44ce37b6-bdca-4a13-8111-3c2a08174e9c" (UID: "44ce37b6-bdca-4a13-8111-3c2a08174e9c"). InnerVolumeSpecName "kube-api-access-hpw49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:32:02.681006 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.680960    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ce37b6-bdca-4a13-8111-3c2a08174e9c-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:02.681006 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.681000    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpw49\" (UniqueName: \"kubernetes.io/projected/44ce37b6-bdca-4a13-8111-3c2a08174e9c-kube-api-access-hpw49\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:02.681006 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.681010    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ce37b6-bdca-4a13-8111-3c2a08174e9c-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:02.984204 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.984173    2584 generic.go:356] "Generic (PLEG): container finished" podID="44ce37b6-bdca-4a13-8111-3c2a08174e9c" containerID="330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c" exitCode=0
Jul 02 19:32:02.984399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.984214    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" event={"ID":"44ce37b6-bdca-4a13-8111-3c2a08174e9c","Type":"ContainerDied","Data":"330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c"}
Jul 02 19:32:02.984399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.984217    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"
Jul 02 19:32:02.984399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.984254    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd" event={"ID":"44ce37b6-bdca-4a13-8111-3c2a08174e9c","Type":"ContainerDied","Data":"c5b3e92529dfe0c7a5a988f12c11c529c1eb8bd52ef0da640cfc16f68b43a1fd"}
Jul 02 19:32:02.984399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.984275    2584 scope.go:117] "RemoveContainer" containerID="330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c"
Jul 02 19:32:02.992665 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.992649    2584 scope.go:117] "RemoveContainer" containerID="330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c"
Jul 02 19:32:02.992924 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:32:02.992906    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c\": container with ID starting with 330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c not found: ID does not exist" containerID="330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c"
Jul 02 19:32:02.993000 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:02.992936    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c"} err="failed to get container status \"330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c\": rpc error: code = NotFound desc = could not find container \"330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c\": container with ID starting with 330232021cbd51e29aecd32657ecde8ca44aaf53640d5af0b76d4fa539e2c45c not found: ID does not exist"
Jul 02 19:32:03.000265 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.000242    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"]
Jul 02 19:32:03.005645 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.005624    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-47a0d36c-6988545b64-rwjcd"]
Jul 02 19:32:03.916234 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.916200    2584 patch_prober.go:28] interesting pod/e2e-mt-47a0d36c-openshift-default-666687647-p4j94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 02 19:32:03.916653 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.916253    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 02 19:32:03.990543 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.990512    2584 generic.go:356] "Generic (PLEG): container finished" podID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerID="e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc" exitCode=0
Jul 02 19:32:03.990714 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.990590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" event={"ID":"9e57796a-9109-41a5-9c3b-f74a7934b6f1","Type":"ContainerDied","Data":"e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"}
Jul 02 19:32:03.991005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.990990    2584 scope.go:117] "RemoveContainer" containerID="e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"
Jul 02 19:32:03.991970 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.991941    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:03.992047 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.992030    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:03.992084 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:03.992074    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:04.777283 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:04.777241    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ce37b6-bdca-4a13-8111-3c2a08174e9c" path="/var/lib/kubelet/pods/44ce37b6-bdca-4a13-8111-3c2a08174e9c/volumes"
Jul 02 19:32:04.997064 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:04.997030    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" event={"ID":"9e57796a-9109-41a5-9c3b-f74a7934b6f1","Type":"ContainerStarted","Data":"4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846"}
Jul 02 19:32:05.785443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:05.785410    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:32:05.785707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:05.785682    2584 patch_prober.go:28] interesting pod/e2e-mt-47a0d36c-openshift-default-666687647-p4j94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 02 19:32:05.785799 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:05.785751    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 02 19:32:06.786062 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:06.786032    2584 patch_prober.go:28] interesting pod/e2e-mt-47a0d36c-openshift-default-666687647-p4j94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 02 19:32:06.786487 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:06.786080    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 02 19:32:07.785663 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:07.785629    2584 patch_prober.go:28] interesting pod/e2e-mt-47a0d36c-openshift-default-666687647-p4j94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 02 19:32:07.785856 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:07.785682    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 02 19:32:08.583133 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:08.583100    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"]
Jul 02 19:32:08.583537 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:08.583295    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy" containerID="cri-o://4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846" gracePeriod=30
Jul 02 19:32:14.235783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.235755    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:32:14.390966 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.390929    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-credential-socket\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.390966 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.390968    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-podinfo\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391001    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istiod-ca-cert\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391026    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-data\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391077    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-socket\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391116    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-certs\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391195    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-token\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391221    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67ctv\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-kube-api-access-67ctv\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391255    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391280    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-envoy\") pod \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\" (UID: \"9e57796a-9109-41a5-9c3b-f74a7934b6f1\") "
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391445    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391489    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391599    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391610    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391630    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.391867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.391645    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.392388 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.392012    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:32:14.393389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.393365    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-kube-api-access-67ctv" (OuterVolumeSpecName: "kube-api-access-67ctv") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "kube-api-access-67ctv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:32:14.393477 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.393369    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:32:14.393477 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.393402    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:32:14.393550 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.393505    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e57796a-9109-41a5-9c3b-f74a7934b6f1" (UID: "9e57796a-9109-41a5-9c3b-f74a7934b6f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:32:14.492023 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.491990    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.492023 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.492022    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67ctv\" (UniqueName: \"kubernetes.io/projected/9e57796a-9109-41a5-9c3b-f74a7934b6f1-kube-api-access-67ctv\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.492217 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.492037    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.492217 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.492047    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.492217 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.492054    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:14.492217 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:14.492067    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e57796a-9109-41a5-9c3b-f74a7934b6f1-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:32:15.041581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.041495    2584 generic.go:356] "Generic (PLEG): container finished" podID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerID="4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846" exitCode=0
Jul 02 19:32:15.041581 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.041545    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"
Jul 02 19:32:15.041776 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.041548    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" event={"ID":"9e57796a-9109-41a5-9c3b-f74a7934b6f1","Type":"ContainerDied","Data":"4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846"}
Jul 02 19:32:15.041776 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.041670    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94" event={"ID":"9e57796a-9109-41a5-9c3b-f74a7934b6f1","Type":"ContainerDied","Data":"47cec0868081f3fe14f2f52468218447ded35c3483305a8021ff3dcd203b13d4"}
Jul 02 19:32:15.041776 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.041698    2584 scope.go:117] "RemoveContainer" containerID="4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846"
Jul 02 19:32:15.050059 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.050045    2584 scope.go:117] "RemoveContainer" containerID="e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"
Jul 02 19:32:15.057528 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.057509    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"]
Jul 02 19:32:15.057945 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.057932    2584 scope.go:117] "RemoveContainer" containerID="4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846"
Jul 02 19:32:15.058192 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:32:15.058177    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846\": container with ID starting with 4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846 not found: ID does not exist" containerID="4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846"
Jul 02 19:32:15.058239 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.058199    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846"} err="failed to get container status \"4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846\": rpc error: code = NotFound desc = could not find container \"4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846\": container with ID starting with 4d72df7b8fcd62ebd7eec39269743ece8a442fcb1271befe239e5df153190846 not found: ID does not exist"
Jul 02 19:32:15.058239 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.058215    2584 scope.go:117] "RemoveContainer" containerID="e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"
Jul 02 19:32:15.058442 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:32:15.058427    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc\": container with ID starting with e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc not found: ID does not exist" containerID="e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"
Jul 02 19:32:15.058599 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.058446    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc"} err="failed to get container status \"e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc\": rpc error: code = NotFound desc = could not find container \"e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc\": container with ID starting with e0d6d04a4717e26c979e7f830f4e5f7556ac675b454192bab9ab86d5a712badc not found: ID does not exist"
Jul 02 19:32:15.061983 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:15.061963    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-47a0d36c-openshift-default-666687647-p4j94"]
Jul 02 19:32:16.777644 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:16.777612    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" path="/var/lib/kubelet/pods/9e57796a-9109-41a5-9c3b-f74a7934b6f1/volumes"
Jul 02 19:32:28.394509 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.394467    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"]
Jul 02 19:32:28.394980 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.394963    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ce37b6-bdca-4a13-8111-3c2a08174e9c" containerName="maas-api"
Jul 02 19:32:28.395025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.394983    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ce37b6-bdca-4a13-8111-3c2a08174e9c" containerName="maas-api"
Jul 02 19:32:28.395025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.395003    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy"
Jul 02 19:32:28.395025 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.395011    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy"
Jul 02 19:32:28.395118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.395030    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy"
Jul 02 19:32:28.395118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.395039    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy"
Jul 02 19:32:28.395189 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.395117    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy"
Jul 02 19:32:28.395189 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.395133    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ce37b6-bdca-4a13-8111-3c2a08174e9c" containerName="maas-api"
Jul 02 19:32:28.399848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.399828    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.402735 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.402713    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-197dee99-openshift-default-dockercfg-f6fbn\""
Jul 02 19:32:28.411999 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.411977    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"]
Jul 02 19:32:28.511495 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511462    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-token\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511495 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-envoy\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511759 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511550    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istiod-ca-cert\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511759 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511579    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-credential-socket\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511759 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511604    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-certs\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511759 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511742    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-podinfo\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511977 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-socket\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511977 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511825    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-data\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.511977 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.511863    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btm8r\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-kube-api-access-btm8r\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612659 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612622    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-podinfo\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612672    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-socket\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612701    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-data\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btm8r\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-kube-api-access-btm8r\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612757    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-token\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-envoy\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.612853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istiod-ca-cert\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612868    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-credential-socket\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.612896    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-certs\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.613163    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-socket\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.613311    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-certs\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.613367    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-data\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613443 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.613404    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-credential-socket\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.613982 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.613956    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istiod-ca-cert\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.615197 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.615178    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-podinfo\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.615285 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.615211    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-envoy\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.620935 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.620909    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-token\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.621240 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.621217    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btm8r\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-kube-api-access-btm8r\") pod \"e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") " pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.711198 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.711118    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:28.848005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.847978    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"]
Jul 02 19:32:28.849867 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:32:28.849839    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e0c50c_5530_40ab_9738_b4bbf3ada1e4.slice/crio-4f03f76f4a0853baea88d89a14c44ea6d7cfef2351d2e00628ae595153b68472 WatchSource:0}: Error finding container 4f03f76f4a0853baea88d89a14c44ea6d7cfef2351d2e00628ae595153b68472: Status 404 returned error can't find the container with id 4f03f76f4a0853baea88d89a14c44ea6d7cfef2351d2e00628ae595153b68472
Jul 02 19:32:28.852089 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.852049    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:28.852203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.852133    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:28.852203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:28.852169    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:29.105823 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:29.105787    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" event={"ID":"23e0c50c-5530-40ab-9738-b4bbf3ada1e4","Type":"ContainerStarted","Data":"e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"}
Jul 02 19:32:29.106016 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:29.105841    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" event={"ID":"23e0c50c-5530-40ab-9738-b4bbf3ada1e4","Type":"ContainerStarted","Data":"4f03f76f4a0853baea88d89a14c44ea6d7cfef2351d2e00628ae595153b68472"}
Jul 02 19:32:29.130937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:29.130888    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" podStartSLOduration=1.130874203 podStartE2EDuration="1.130874203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:32:29.129847383 +0000 UTC m=+1728.984779152" watchObservedRunningTime="2026-07-02 19:32:29.130874203 +0000 UTC m=+1728.985805973"
Jul 02 19:32:29.711750 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:29.711713    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:29.716528 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:29.716503    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:30.110438 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:30.110410    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:30.111470 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:30.111453    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:31.327528 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.327494    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"]
Jul 02 19:32:31.328121 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.328101    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e57796a-9109-41a5-9c3b-f74a7934b6f1" containerName="istio-proxy"
Jul 02 19:32:31.331109 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.331088    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.331729 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.331704    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj9bk\" (UniqueName: \"kubernetes.io/projected/fbef539f-2f76-4a76-a48e-3080e71d8632-kube-api-access-qj9bk\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.331827 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.331745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.331827 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.331818    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbef539f-2f76-4a76-a48e-3080e71d8632-openshift-service-ca\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.333449 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.333429    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-197dee99-serving-cert\""
Jul 02 19:32:31.346499 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.346477    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"]
Jul 02 19:32:31.432921 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.432878    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbef539f-2f76-4a76-a48e-3080e71d8632-openshift-service-ca\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.433111 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.432970    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9bk\" (UniqueName: \"kubernetes.io/projected/fbef539f-2f76-4a76-a48e-3080e71d8632-kube-api-access-qj9bk\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.433111 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.433011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.433229 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:32:31.433130    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-serving-cert: secret "maas-api-e2e-mt-197dee99-serving-cert" not found
Jul 02 19:32:31.433229 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:32:31.433201    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls podName:fbef539f-2f76-4a76-a48e-3080e71d8632 nodeName:}" failed. No retries permitted until 2026-07-02 19:32:31.933177659 +0000 UTC m=+1731.788109418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls") pod "maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" (UID: "fbef539f-2f76-4a76-a48e-3080e71d8632") : secret "maas-api-e2e-mt-197dee99-serving-cert" not found
Jul 02 19:32:31.433752 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.433727    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbef539f-2f76-4a76-a48e-3080e71d8632-openshift-service-ca\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.450465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.450432    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9bk\" (UniqueName: \"kubernetes.io/projected/fbef539f-2f76-4a76-a48e-3080e71d8632-kube-api-access-qj9bk\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.936563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.936527    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.938915 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.938890    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls\") pod \"maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:31.941782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:31.941759    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:32.102867 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:32.102824    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"]
Jul 02 19:32:32.114154 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:32:32.114129    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbef539f_2f76_4a76_a48e_3080e71d8632.slice/crio-65e6dde2f523a325773044ec0bb18a1ef74f3693fd8caec7b5c70b23438ed1b3 WatchSource:0}: Error finding container 65e6dde2f523a325773044ec0bb18a1ef74f3693fd8caec7b5c70b23438ed1b3: Status 404 returned error can't find the container with id 65e6dde2f523a325773044ec0bb18a1ef74f3693fd8caec7b5c70b23438ed1b3
Jul 02 19:32:32.121754 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:32.121727    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" event={"ID":"fbef539f-2f76-4a76-a48e-3080e71d8632","Type":"ContainerStarted","Data":"65e6dde2f523a325773044ec0bb18a1ef74f3693fd8caec7b5c70b23438ed1b3"}
Jul 02 19:32:33.127233 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:33.127199    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" event={"ID":"fbef539f-2f76-4a76-a48e-3080e71d8632","Type":"ContainerStarted","Data":"a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a"}
Jul 02 19:32:33.127615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:33.127301    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:33.154569 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:33.154494    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" podStartSLOduration=1.736325146 podStartE2EDuration="2.154477238s" totalImagesPullingTime="418.152092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:32:33.152604667 +0000 UTC m=+1733.007536437" watchObservedRunningTime="2026-07-02 19:32:33.154477238 +0000 UTC m=+1733.009409009"
Jul 02 19:32:39.148270 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:39.147601    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:32:44.184858 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:44.184769    2584 generic.go:356] "Generic (PLEG): container finished" podID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerID="e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3" exitCode=0
Jul 02 19:32:44.184858 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:44.184814    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" event={"ID":"23e0c50c-5530-40ab-9738-b4bbf3ada1e4","Type":"ContainerDied","Data":"e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"}
Jul 02 19:32:44.185280 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:44.185160    2584 scope.go:117] "RemoveContainer" containerID="e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"
Jul 02 19:32:44.186135 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:44.186106    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:44.186211 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:44.186172    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:44.186211 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:44.186201    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:32:45.111050 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:45.111013    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:45.191101 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:45.191067    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" event={"ID":"23e0c50c-5530-40ab-9738-b4bbf3ada1e4","Type":"ContainerStarted","Data":"c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd"}
Jul 02 19:32:45.711252 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:45.711216    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:45.712388 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:45.712364    2584 patch_prober.go:28] interesting pod/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 02 19:32:45.712450 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:45.712414    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 02 19:32:46.711971 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:46.711936    2584 patch_prober.go:28] interesting pod/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 02 19:32:46.712390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:46.711997    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 02 19:32:47.711745 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:47.711705    2584 patch_prober.go:28] interesting pod/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 02 19:32:47.711914 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:47.711772    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 02 19:32:48.711807 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:48.711753    2584 patch_prober.go:28] interesting pod/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 02 19:32:48.712195 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:48.711808    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 02 19:32:49.715510 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:49.715480    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:49.715887 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:49.715821    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:32:49.716657 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:32:49.716635    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:33:13.555235 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.555202    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"]
Jul 02 19:33:13.555818 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.555459    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" podUID="fbef539f-2f76-4a76-a48e-3080e71d8632" containerName="maas-api" containerID="cri-o://a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a" gracePeriod=30
Jul 02 19:33:13.801604 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.801578    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:33:13.912712 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.912676    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls\") pod \"fbef539f-2f76-4a76-a48e-3080e71d8632\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") "
Jul 02 19:33:13.912918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.912721    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj9bk\" (UniqueName: \"kubernetes.io/projected/fbef539f-2f76-4a76-a48e-3080e71d8632-kube-api-access-qj9bk\") pod \"fbef539f-2f76-4a76-a48e-3080e71d8632\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") "
Jul 02 19:33:13.912918 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.912780    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbef539f-2f76-4a76-a48e-3080e71d8632-openshift-service-ca\") pod \"fbef539f-2f76-4a76-a48e-3080e71d8632\" (UID: \"fbef539f-2f76-4a76-a48e-3080e71d8632\") "
Jul 02 19:33:13.913188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.913158    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbef539f-2f76-4a76-a48e-3080e71d8632-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbef539f-2f76-4a76-a48e-3080e71d8632" (UID: "fbef539f-2f76-4a76-a48e-3080e71d8632"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:33:13.914934 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.914912    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbef539f-2f76-4a76-a48e-3080e71d8632-kube-api-access-qj9bk" (OuterVolumeSpecName: "kube-api-access-qj9bk") pod "fbef539f-2f76-4a76-a48e-3080e71d8632" (UID: "fbef539f-2f76-4a76-a48e-3080e71d8632"). InnerVolumeSpecName "kube-api-access-qj9bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:33:13.915007 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:13.914930    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbef539f-2f76-4a76-a48e-3080e71d8632" (UID: "fbef539f-2f76-4a76-a48e-3080e71d8632"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:33:14.014425 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.014390    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbef539f-2f76-4a76-a48e-3080e71d8632-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:33:14.014425 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.014416    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbef539f-2f76-4a76-a48e-3080e71d8632-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:33:14.014425 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.014425    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj9bk\" (UniqueName: \"kubernetes.io/projected/fbef539f-2f76-4a76-a48e-3080e71d8632-kube-api-access-qj9bk\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:33:14.323497 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.323390    2584 generic.go:356] "Generic (PLEG): container finished" podID="fbef539f-2f76-4a76-a48e-3080e71d8632" containerID="a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a" exitCode=0
Jul 02 19:33:14.323497 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.323434    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"
Jul 02 19:33:14.323497 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.323467    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" event={"ID":"fbef539f-2f76-4a76-a48e-3080e71d8632","Type":"ContainerDied","Data":"a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a"}
Jul 02 19:33:14.323748 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.323519    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd" event={"ID":"fbef539f-2f76-4a76-a48e-3080e71d8632","Type":"ContainerDied","Data":"65e6dde2f523a325773044ec0bb18a1ef74f3693fd8caec7b5c70b23438ed1b3"}
Jul 02 19:33:14.323748 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.323541    2584 scope.go:117] "RemoveContainer" containerID="a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a"
Jul 02 19:33:14.332627 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.332613    2584 scope.go:117] "RemoveContainer" containerID="a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a"
Jul 02 19:33:14.332863 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:33:14.332848    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a\": container with ID starting with a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a not found: ID does not exist" containerID="a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a"
Jul 02 19:33:14.332941 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.332870    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a"} err="failed to get container status \"a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a\": rpc error: code = NotFound desc = could not find container \"a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a\": container with ID starting with a879f8ec8a0c86414925d2a6c0e4467b5cd4278feabeaff8e2a3e36318ecfb3a not found: ID does not exist"
Jul 02 19:33:14.347695 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.347667    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"]
Jul 02 19:33:14.351824 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.351805    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-197dee99-6d6777bd45-hwvrd"]
Jul 02 19:33:14.777292 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:33:14.777259    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbef539f-2f76-4a76-a48e-3080e71d8632" path="/var/lib/kubelet/pods/fbef539f-2f76-4a76-a48e-3080e71d8632/volumes"
Jul 02 19:34:07.018431 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:07.018379    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"]
Jul 02 19:34:07.019247 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:07.018646    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy" containerID="cri-o://c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd" gracePeriod=30
Jul 02 19:34:12.277804 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.277739    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:34:12.319226 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319197    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-certs\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319260    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-podinfo\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319285    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btm8r\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-kube-api-access-btm8r\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319313    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istiod-ca-cert\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319384    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-credential-socket\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319630 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319412    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-socket\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319630 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319445    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-envoy\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319630 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319473    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-data\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319630 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319491    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:34:12.319630 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319506    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-token\") pod \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\" (UID: \"23e0c50c-5530-40ab-9738-b4bbf3ada1e4\") "
Jul 02 19:34:12.319885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319660    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:34:12.319885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319686    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:34:12.319885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319771    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:34:12.319885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319875    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.320119 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319896    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.320119 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.319912    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.320459 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.320435    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:34:12.321686 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.321662    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:34:12.321922 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.321907    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:34:12.322070 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.322054    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:34:12.322143 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.322127    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-kube-api-access-btm8r" (OuterVolumeSpecName: "kube-api-access-btm8r") pod "23e0c50c-5530-40ab-9738-b4bbf3ada1e4" (UID: "23e0c50c-5530-40ab-9738-b4bbf3ada1e4"). InnerVolumeSpecName "kube-api-access-btm8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:34:12.420322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.420290    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.420322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.420315    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.420322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.420327    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btm8r\" (UniqueName: \"kubernetes.io/projected/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-kube-api-access-btm8r\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.420553 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.420355    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.420553 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.420364    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.420553 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.420372    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e0c50c-5530-40ab-9738-b4bbf3ada1e4-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:34:12.586681 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.586645    2584 generic.go:356] "Generic (PLEG): container finished" podID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerID="c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd" exitCode=0
Jul 02 19:34:12.586833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.586692    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"
Jul 02 19:34:12.586833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.586691    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" event={"ID":"23e0c50c-5530-40ab-9738-b4bbf3ada1e4","Type":"ContainerDied","Data":"c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd"}
Jul 02 19:34:12.586934 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.586867    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz" event={"ID":"23e0c50c-5530-40ab-9738-b4bbf3ada1e4","Type":"ContainerDied","Data":"4f03f76f4a0853baea88d89a14c44ea6d7cfef2351d2e00628ae595153b68472"}
Jul 02 19:34:12.586934 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.586905    2584 scope.go:117] "RemoveContainer" containerID="c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd"
Jul 02 19:34:12.596008 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.595991    2584 scope.go:117] "RemoveContainer" containerID="e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"
Jul 02 19:34:12.603797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.603782    2584 scope.go:117] "RemoveContainer" containerID="c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd"
Jul 02 19:34:12.604008 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:34:12.603991    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd\": container with ID starting with c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd not found: ID does not exist" containerID="c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd"
Jul 02 19:34:12.604064 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.604016    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd"} err="failed to get container status \"c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd\": rpc error: code = NotFound desc = could not find container \"c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd\": container with ID starting with c15cdc1883eb8ed57c80d7ab355539dc62ddcc240be60c1e81116db192fc8bbd not found: ID does not exist"
Jul 02 19:34:12.604064 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.604030    2584 scope.go:117] "RemoveContainer" containerID="e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"
Jul 02 19:34:12.604234 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:34:12.604220    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3\": container with ID starting with e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3 not found: ID does not exist" containerID="e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"
Jul 02 19:34:12.604274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.604235    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3"} err="failed to get container status \"e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3\": rpc error: code = NotFound desc = could not find container \"e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3\": container with ID starting with e6665e726016c6d53d8cbf5fb265799d645be86fb6eb441dce38f1ac57a70ad3 not found: ID does not exist"
Jul 02 19:34:12.607407 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.607386    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"]
Jul 02 19:34:12.614200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.614178    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-197dee99-openshift-default-797d5676f9-vs4jz"]
Jul 02 19:34:12.776549 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:34:12.776519    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" path="/var/lib/kubelet/pods/23e0c50c-5530-40ab-9738-b4bbf3ada1e4/volumes"
Jul 02 19:35:01.951607 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951576    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"]
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951897    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbef539f-2f76-4a76-a48e-3080e71d8632" containerName="maas-api"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951908    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbef539f-2f76-4a76-a48e-3080e71d8632" containerName="maas-api"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951919    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951925    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951930    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951936    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951987    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy"
Jul 02 19:35:01.951992 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.951996    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbef539f-2f76-4a76-a48e-3080e71d8632" containerName="maas-api"
Jul 02 19:35:01.954815 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.954796    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:01.957158 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.957136    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-fd9e49-openshift-default-dockercfg-vt5f9\""
Jul 02 19:35:01.972666 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:01.972617    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"]
Jul 02 19:35:02.129956 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.129875    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-token\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130127 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.129951    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-socket\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130127 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130031    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-podinfo\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130127 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130080    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istiod-ca-cert\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-certs\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-envoy\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130183    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-data\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130213    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fst9g\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-kube-api-access-fst9g\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.130302 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.130248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-credential-socket\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231460 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-certs\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231460 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231417    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-envoy\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231460 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-data\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231465    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fst9g\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-kube-api-access-fst9g\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231494    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-credential-socket\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231541    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-token\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231589    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-socket\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231641    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-podinfo\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.231724 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istiod-ca-cert\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.232010 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231828    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-certs\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.232010 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.231891    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-data\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.232090 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.232074    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-credential-socket\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.232135 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.232090    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-socket\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.232441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.232423    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istiod-ca-cert\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.233858 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.233827    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-envoy\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.234019 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.234002    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-podinfo\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.239250 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.239222    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fst9g\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-kube-api-access-fst9g\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.239477 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.239459    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-token\") pod \"e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") " pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.270764 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.270742    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:02.411224 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.411193    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"]
Jul 02 19:35:02.414550 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:35:02.414522    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84617ca8_7c00_4f86_99a5_5dd1b9d72329.slice/crio-239f6563c27b40625ed7cb67af467c45b0459b5b9f16a9448859d6ddd6eba609 WatchSource:0}: Error finding container 239f6563c27b40625ed7cb67af467c45b0459b5b9f16a9448859d6ddd6eba609: Status 404 returned error can't find the container with id 239f6563c27b40625ed7cb67af467c45b0459b5b9f16a9448859d6ddd6eba609
Jul 02 19:35:02.416768 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.416734    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:02.416844 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.416800    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:02.416844 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.416828    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:02.808856 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.808822    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" event={"ID":"84617ca8-7c00-4f86-99a5-5dd1b9d72329","Type":"ContainerStarted","Data":"20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"}
Jul 02 19:35:02.809035 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.808876    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" event={"ID":"84617ca8-7c00-4f86-99a5-5dd1b9d72329","Type":"ContainerStarted","Data":"239f6563c27b40625ed7cb67af467c45b0459b5b9f16a9448859d6ddd6eba609"}
Jul 02 19:35:02.829169 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:02.829125    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podStartSLOduration=1.82911075 podStartE2EDuration="1.82911075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:35:02.827230305 +0000 UTC m=+1882.682162086" watchObservedRunningTime="2026-07-02 19:35:02.82911075 +0000 UTC m=+1882.684042520"
Jul 02 19:35:03.271789 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:03.271752    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:03.276986 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:03.276961    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:03.815188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:03.815156    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:03.816030 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:03.816006    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:05.102080 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.102048    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"]
Jul 02 19:35:05.102550 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.102433    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="23e0c50c-5530-40ab-9738-b4bbf3ada1e4" containerName="istio-proxy"
Jul 02 19:35:05.105429 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.105408    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.107929 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.107908    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fd9e49-serving-cert\""
Jul 02 19:35:05.118785 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.118763    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"]
Jul 02 19:35:05.256952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.256922    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.257134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.256961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-kube-api-access-r4gwz\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.257134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.256990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.358362 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.358265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.358521 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.358381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.358521 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.358409    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-kube-api-access-r4gwz\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.358594 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:35:05.358543    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-serving-cert: secret "maas-api-e2e-inf-a-fd9e49-serving-cert" not found
Jul 02 19:35:05.358665 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:35:05.358653    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls podName:b235c0f0-59c9-415c-a0c9-8dce9fc51f18 nodeName:}" failed. No retries permitted until 2026-07-02 19:35:05.858631438 +0000 UTC m=+1885.713563185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls") pod "maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" (UID: "b235c0f0-59c9-415c-a0c9-8dce9fc51f18") : secret "maas-api-e2e-inf-a-fd9e49-serving-cert" not found
Jul 02 19:35:05.358958 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.358942    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.369849 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.369825    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-kube-api-access-r4gwz\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.863777 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.863740    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:05.866156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:05.866126    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:06.016329 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:06.016297    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:06.161620 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:06.161586    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"]
Jul 02 19:35:06.166860 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:35:06.166832    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb235c0f0_59c9_415c_a0c9_8dce9fc51f18.slice/crio-528c46760f0520c57be964cc6a3c898216c3b621ff5516ef97630e1413fcf785 WatchSource:0}: Error finding container 528c46760f0520c57be964cc6a3c898216c3b621ff5516ef97630e1413fcf785: Status 404 returned error can't find the container with id 528c46760f0520c57be964cc6a3c898216c3b621ff5516ef97630e1413fcf785
Jul 02 19:35:06.168291 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:06.168274    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:35:06.830124 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:06.830101    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" event={"ID":"b235c0f0-59c9-415c-a0c9-8dce9fc51f18","Type":"ContainerStarted","Data":"528c46760f0520c57be964cc6a3c898216c3b621ff5516ef97630e1413fcf785"}
Jul 02 19:35:07.836377 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:07.836323    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" event={"ID":"b235c0f0-59c9-415c-a0c9-8dce9fc51f18","Type":"ContainerStarted","Data":"e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c"}
Jul 02 19:35:07.836831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:07.836408    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:07.854449 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:07.854406    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" podStartSLOduration=2.235642949 podStartE2EDuration="2.854392128s" totalImagesPullingTime="618.749179ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:35:07.851979255 +0000 UTC m=+1887.706911026" watchObservedRunningTime="2026-07-02 19:35:07.854392128 +0000 UTC m=+1887.709323898"
Jul 02 19:35:10.030073 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.030043    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"]
Jul 02 19:35:10.033826 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.033802    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.036633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.036611    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-78744c-openshift-default-dockercfg-tmcgq\""
Jul 02 19:35:10.045026 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.045004    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"]
Jul 02 19:35:10.097868 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.097835    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pzcl\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-kube-api-access-7pzcl\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098029 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.097876    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istiod-ca-cert\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098029 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.097944    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-podinfo\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098029 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.097964    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-credential-socket\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098029 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.097986    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-envoy\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098029 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.098004    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-certs\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098029 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.098026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-token\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098251 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.098058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-data\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.098251 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.098098    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-socket\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198589 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-socket\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzcl\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-kube-api-access-7pzcl\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istiod-ca-cert\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198661    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-podinfo\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198677    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-credential-socket\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198702    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-envoy\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.198783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-certs\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199114 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-token\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199114 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.198935    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-data\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199114 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.199058    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-socket\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.199234    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-credential-socket\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.199246    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-certs\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.199263    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-data\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.199738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.199719    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istiod-ca-cert\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.200984 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.200959    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-envoy\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.201353 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.201316    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-podinfo\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.206604 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.206572    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-token\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.207460 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.207436    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pzcl\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-kube-api-access-7pzcl\") pod \"e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") " pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.344308 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.344283    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:10.480624 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.480599    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"]
Jul 02 19:35:10.482824 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:35:10.482796    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b3e2d2_fdd2_4e58_97a9_165ac1bb4e55.slice/crio-e38876976697ba6c7603fe89cafeaebfd9b331fa0bbdcd048e6cf1436aeaa73a WatchSource:0}: Error finding container e38876976697ba6c7603fe89cafeaebfd9b331fa0bbdcd048e6cf1436aeaa73a: Status 404 returned error can't find the container with id e38876976697ba6c7603fe89cafeaebfd9b331fa0bbdcd048e6cf1436aeaa73a
Jul 02 19:35:10.485483 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.485450    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:10.485580 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.485520    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:10.485580 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.485551    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:10.851721 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.851685    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" event={"ID":"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55","Type":"ContainerStarted","Data":"b61e76dae1347c1bf304563512aefd3337786f170b285365fef1aa16dc4c71ef"}
Jul 02 19:35:10.851902 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.851746    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" event={"ID":"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55","Type":"ContainerStarted","Data":"e38876976697ba6c7603fe89cafeaebfd9b331fa0bbdcd048e6cf1436aeaa73a"}
Jul 02 19:35:10.873527 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:10.873477    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podStartSLOduration=0.873458274 podStartE2EDuration="873.458274ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:35:10.872416061 +0000 UTC m=+1890.727347841" watchObservedRunningTime="2026-07-02 19:35:10.873458274 +0000 UTC m=+1890.728390046"
Jul 02 19:35:11.344921 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:11.344890    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:11.349653 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:11.349624    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:11.856187 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:11.856156    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:11.857077 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:11.857056    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:13.005100 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.005060    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"]
Jul 02 19:35:13.008629 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.008606    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.011222 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.011202    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-78744c-serving-cert\""
Jul 02 19:35:13.020570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.020547    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"]
Jul 02 19:35:13.020746 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.020718    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88c0176-8a7a-4d99-8371-33c290cbdbd1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.020835 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.020763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.020894 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.020869    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8zkk\" (UniqueName: \"kubernetes.io/projected/d88c0176-8a7a-4d99-8371-33c290cbdbd1-kube-api-access-q8zkk\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.122490 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.122455    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8zkk\" (UniqueName: \"kubernetes.io/projected/d88c0176-8a7a-4d99-8371-33c290cbdbd1-kube-api-access-q8zkk\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.122686 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.122558    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88c0176-8a7a-4d99-8371-33c290cbdbd1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.122686 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.122587    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.122823 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:35:13.122715    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-serving-cert: secret "maas-api-e2e-inf-b-78744c-serving-cert" not found
Jul 02 19:35:13.122823 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:35:13.122783    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls podName:d88c0176-8a7a-4d99-8371-33c290cbdbd1 nodeName:}" failed. No retries permitted until 2026-07-02 19:35:13.622761399 +0000 UTC m=+1893.477693171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls") pod "maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" (UID: "d88c0176-8a7a-4d99-8371-33c290cbdbd1") : secret "maas-api-e2e-inf-b-78744c-serving-cert" not found
Jul 02 19:35:13.123817 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.123788    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88c0176-8a7a-4d99-8371-33c290cbdbd1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.147123 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.147087    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8zkk\" (UniqueName: \"kubernetes.io/projected/d88c0176-8a7a-4d99-8371-33c290cbdbd1-kube-api-access-q8zkk\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.627127 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.627099    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.638128 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.638078    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-78744c-865f67dc88-d42sw\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:13.853766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.853703    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:35:13.920584 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:13.920503    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:14.114802 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:14.114778    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"]
Jul 02 19:35:14.119303 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:35:14.119272    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88c0176_8a7a_4d99_8371_33c290cbdbd1.slice/crio-721207682b8a42ac1306ebaca911b746942e21c2e6ce252ec990d819f9e4ecb0 WatchSource:0}: Error finding container 721207682b8a42ac1306ebaca911b746942e21c2e6ce252ec990d819f9e4ecb0: Status 404 returned error can't find the container with id 721207682b8a42ac1306ebaca911b746942e21c2e6ce252ec990d819f9e4ecb0
Jul 02 19:35:14.871505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:14.871462    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" event={"ID":"d88c0176-8a7a-4d99-8371-33c290cbdbd1","Type":"ContainerStarted","Data":"e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac"}
Jul 02 19:35:14.871704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:14.871521    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" event={"ID":"d88c0176-8a7a-4d99-8371-33c290cbdbd1","Type":"ContainerStarted","Data":"721207682b8a42ac1306ebaca911b746942e21c2e6ce252ec990d819f9e4ecb0"}
Jul 02 19:35:14.871704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:14.871550    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:14.894483 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:14.894432    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" podStartSLOduration=2.5283107449999997 podStartE2EDuration="2.894417579s" totalImagesPullingTime="366.106834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:35:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:35:14.89198361 +0000 UTC m=+1894.746915381" watchObservedRunningTime="2026-07-02 19:35:14.894417579 +0000 UTC m=+1894.749349355"
Jul 02 19:35:19.856602 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:19.856551    2584 patch_prober.go:28] interesting pod/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 02 19:35:19.857222 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:19.857181    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 02 19:35:20.882552 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.882524    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:35:20.903011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.902983    2584 generic.go:356] "Generic (PLEG): container finished" podID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerID="20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d" exitCode=0
Jul 02 19:35:20.903159 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.903056    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" event={"ID":"84617ca8-7c00-4f86-99a5-5dd1b9d72329","Type":"ContainerDied","Data":"20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"}
Jul 02 19:35:20.903471 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.903454    2584 scope.go:117] "RemoveContainer" containerID="20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"
Jul 02 19:35:20.904441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.904405    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:20.904507 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.904477    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:20.904548 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:20.904524    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:21.909928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:21.909897    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" event={"ID":"84617ca8-7c00-4f86-99a5-5dd1b9d72329","Type":"ContainerStarted","Data":"5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb"}
Jul 02 19:35:22.272323 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:22.272215    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:22.272525 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:22.272501    2584 patch_prober.go:28] interesting pod/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 02 19:35:22.272595 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:22.272558    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 02 19:35:23.272070 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:23.272029    2584 patch_prober.go:28] interesting pod/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 02 19:35:23.273115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:23.272675    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 02 19:35:24.272142 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:24.272056    2584 patch_prober.go:28] interesting pod/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 02 19:35:24.272518 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:24.272130    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 02 19:35:25.271420 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:25.271378    2584 patch_prober.go:28] interesting pod/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 02 19:35:25.271612 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:25.271448    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 02 19:35:26.274315 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:26.274279    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:26.274789 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:26.274620    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:26.275392 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:26.275373    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:35:27.085619 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.085098    2584 patch_prober.go:28] interesting pod/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:51886->10.134.0.50:15021: read: connection reset by peer" start-of-body=
Jul 02 19:35:27.085619 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.085157    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:51886->10.134.0.50:15021: read: connection reset by peer"
Jul 02 19:35:27.938782 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.938750    2584 generic.go:356] "Generic (PLEG): container finished" podID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerID="b61e76dae1347c1bf304563512aefd3337786f170b285365fef1aa16dc4c71ef" exitCode=0
Jul 02 19:35:27.939268 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.938831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" event={"ID":"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55","Type":"ContainerDied","Data":"b61e76dae1347c1bf304563512aefd3337786f170b285365fef1aa16dc4c71ef"}
Jul 02 19:35:27.939421 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.939392    2584 scope.go:117] "RemoveContainer" containerID="b61e76dae1347c1bf304563512aefd3337786f170b285365fef1aa16dc4c71ef"
Jul 02 19:35:27.940297 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.940270    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:27.940374 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.940334    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:27.940418 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:27.940388    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:35:28.948179 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:28.948138    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" event={"ID":"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55","Type":"ContainerStarted","Data":"605dd244d58ea894e7bfd670070a540918f0df02cde3741da96323a72bd2f890"}
Jul 02 19:35:29.344678 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:29.344634    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:29.345091 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:29.345061    2584 patch_prober.go:28] interesting pod/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 02 19:35:29.345204 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:29.345134    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 02 19:35:30.344853 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:30.344808    2584 patch_prober.go:28] interesting pod/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 02 19:35:30.345380 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:30.344876    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 02 19:35:31.345275 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:31.345230    2584 patch_prober.go:28] interesting pod/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 02 19:35:31.345817 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:31.345301    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 02 19:35:32.345026 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:32.344981    2584 patch_prober.go:28] interesting pod/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 02 19:35:32.345243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:32.345047    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 02 19:35:33.348627 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:33.348596    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:33.349036 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:33.348851    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:35:33.349498 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:35:33.349482    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:36:33.479295 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.479209    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"]
Jul 02 19:36:33.479888 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.479517    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" podUID="b235c0f0-59c9-415c-a0c9-8dce9fc51f18" containerName="maas-api" containerID="cri-o://e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c" gracePeriod=30
Jul 02 19:36:33.732873 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.732811    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:36:33.875457 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.875415    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-openshift-service-ca\") pod \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") "
Jul 02 19:36:33.875634 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.875487    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls\") pod \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") "
Jul 02 19:36:33.875634 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.875581    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-kube-api-access-r4gwz\") pod \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\" (UID: \"b235c0f0-59c9-415c-a0c9-8dce9fc51f18\") "
Jul 02 19:36:33.875866 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.875846    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b235c0f0-59c9-415c-a0c9-8dce9fc51f18" (UID: "b235c0f0-59c9-415c-a0c9-8dce9fc51f18"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:36:33.877645 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.877614    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b235c0f0-59c9-415c-a0c9-8dce9fc51f18" (UID: "b235c0f0-59c9-415c-a0c9-8dce9fc51f18"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:36:33.877760 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.877660    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-kube-api-access-r4gwz" (OuterVolumeSpecName: "kube-api-access-r4gwz") pod "b235c0f0-59c9-415c-a0c9-8dce9fc51f18" (UID: "b235c0f0-59c9-415c-a0c9-8dce9fc51f18"). InnerVolumeSpecName "kube-api-access-r4gwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:36:33.976242 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.976204    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:36:33.976242 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.976236    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:36:33.976242 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:33.976245    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/b235c0f0-59c9-415c-a0c9-8dce9fc51f18-kube-api-access-r4gwz\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:36:34.258201 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.258165    2584 generic.go:356] "Generic (PLEG): container finished" podID="b235c0f0-59c9-415c-a0c9-8dce9fc51f18" containerID="e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c" exitCode=0
Jul 02 19:36:34.258390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.258211    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"
Jul 02 19:36:34.258390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.258254    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" event={"ID":"b235c0f0-59c9-415c-a0c9-8dce9fc51f18","Type":"ContainerDied","Data":"e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c"}
Jul 02 19:36:34.258390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.258312    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx" event={"ID":"b235c0f0-59c9-415c-a0c9-8dce9fc51f18","Type":"ContainerDied","Data":"528c46760f0520c57be964cc6a3c898216c3b621ff5516ef97630e1413fcf785"}
Jul 02 19:36:34.258390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.258354    2584 scope.go:117] "RemoveContainer" containerID="e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c"
Jul 02 19:36:34.267052 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.267034    2584 scope.go:117] "RemoveContainer" containerID="e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c"
Jul 02 19:36:34.267307 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:36:34.267287    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c\": container with ID starting with e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c not found: ID does not exist" containerID="e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c"
Jul 02 19:36:34.267363 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.267315    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c"} err="failed to get container status \"e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c\": rpc error: code = NotFound desc = could not find container \"e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c\": container with ID starting with e65214e702f9b0a46e53c9af7ccb361d9f9eccc572d127c14afa45d9414f579c not found: ID does not exist"
Jul 02 19:36:34.278635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.278607    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"]
Jul 02 19:36:34.283131 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.283106    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd9e49-5ff4cff9cf-g24cx"]
Jul 02 19:36:34.777458 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:36:34.777427    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b235c0f0-59c9-415c-a0c9-8dce9fc51f18" path="/var/lib/kubelet/pods/b235c0f0-59c9-415c-a0c9-8dce9fc51f18/volumes"
Jul 02 19:37:15.428978 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:15.428946    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"]
Jul 02 19:37:15.429618 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:15.429214    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy" containerID="cri-o://5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb" gracePeriod=30
Jul 02 19:37:16.559231 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.559197    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"]
Jul 02 19:37:16.559707 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.559455    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" podUID="d88c0176-8a7a-4d99-8371-33c290cbdbd1" containerName="maas-api" containerID="cri-o://e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac" gracePeriod=30
Jul 02 19:37:16.804669 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.804646    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:37:16.966304 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.966270    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8zkk\" (UniqueName: \"kubernetes.io/projected/d88c0176-8a7a-4d99-8371-33c290cbdbd1-kube-api-access-q8zkk\") pod \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") "
Jul 02 19:37:16.966503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.966363    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls\") pod \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") "
Jul 02 19:37:16.966503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.966402    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88c0176-8a7a-4d99-8371-33c290cbdbd1-openshift-service-ca\") pod \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\" (UID: \"d88c0176-8a7a-4d99-8371-33c290cbdbd1\") "
Jul 02 19:37:16.966863 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.966831    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d88c0176-8a7a-4d99-8371-33c290cbdbd1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d88c0176-8a7a-4d99-8371-33c290cbdbd1" (UID: "d88c0176-8a7a-4d99-8371-33c290cbdbd1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:37:16.968444 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.968416    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d88c0176-8a7a-4d99-8371-33c290cbdbd1" (UID: "d88c0176-8a7a-4d99-8371-33c290cbdbd1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:37:16.968534 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:16.968485    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88c0176-8a7a-4d99-8371-33c290cbdbd1-kube-api-access-q8zkk" (OuterVolumeSpecName: "kube-api-access-q8zkk") pod "d88c0176-8a7a-4d99-8371-33c290cbdbd1" (UID: "d88c0176-8a7a-4d99-8371-33c290cbdbd1"). InnerVolumeSpecName "kube-api-access-q8zkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:17.068107 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.068073    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88c0176-8a7a-4d99-8371-33c290cbdbd1-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:17.068107 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.068101    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88c0176-8a7a-4d99-8371-33c290cbdbd1-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:17.068107 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.068111    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8zkk\" (UniqueName: \"kubernetes.io/projected/d88c0176-8a7a-4d99-8371-33c290cbdbd1-kube-api-access-q8zkk\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:17.450219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.450187    2584 generic.go:356] "Generic (PLEG): container finished" podID="d88c0176-8a7a-4d99-8371-33c290cbdbd1" containerID="e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac" exitCode=0
Jul 02 19:37:17.450411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.450229    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"
Jul 02 19:37:17.450411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.450273    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" event={"ID":"d88c0176-8a7a-4d99-8371-33c290cbdbd1","Type":"ContainerDied","Data":"e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac"}
Jul 02 19:37:17.450411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.450325    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw" event={"ID":"d88c0176-8a7a-4d99-8371-33c290cbdbd1","Type":"ContainerDied","Data":"721207682b8a42ac1306ebaca911b746942e21c2e6ce252ec990d819f9e4ecb0"}
Jul 02 19:37:17.450411 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.450366    2584 scope.go:117] "RemoveContainer" containerID="e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac"
Jul 02 19:37:17.459773 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.459753    2584 scope.go:117] "RemoveContainer" containerID="e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac"
Jul 02 19:37:17.460033 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:37:17.460015    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac\": container with ID starting with e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac not found: ID does not exist" containerID="e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac"
Jul 02 19:37:17.460078 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.460042    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac"} err="failed to get container status \"e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac\": rpc error: code = NotFound desc = could not find container \"e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac\": container with ID starting with e6dbe38984b312f98182c5d074c9d028e3bc0d3679a19015ba218f1248d51fac not found: ID does not exist"
Jul 02 19:37:17.477800 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.477776    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"]
Jul 02 19:37:17.490121 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:17.490083    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-78744c-865f67dc88-d42sw"]
Jul 02 19:37:18.777205 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:18.777169    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d88c0176-8a7a-4d99-8371-33c290cbdbd1" path="/var/lib/kubelet/pods/d88c0176-8a7a-4d99-8371-33c290cbdbd1/volumes"
Jul 02 19:37:20.682582 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.682551    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:37:20.703011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.702949    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-token\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.703011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.702987    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-podinfo\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.703184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703028    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-socket\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.703184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703077    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-credential-socket\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.703301 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703217    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istiod-ca-cert\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.703301 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703263    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-certs\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.703301 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703293    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-envoy\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703305    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703317    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-data\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703329    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703362    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fst9g\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-kube-api-access-fst9g\") pod \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\" (UID: \"84617ca8-7c00-4f86-99a5-5dd1b9d72329\") "
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703474    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703723    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703812    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703865    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.703884    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.705414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.704481    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-data" (OuterVolumeSpecName: "istio-data") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:20.705985 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.705571    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-token" (OuterVolumeSpecName: "istio-token") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:20.706112 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.706052    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-kube-api-access-fst9g" (OuterVolumeSpecName: "kube-api-access-fst9g") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "kube-api-access-fst9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:20.706450 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.706424    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:20.706551 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.706501    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84617ca8-7c00-4f86-99a5-5dd1b9d72329" (UID: "84617ca8-7c00-4f86-99a5-5dd1b9d72329"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:37:20.805163 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.805136    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.805163 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.805158    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.805163 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.805167    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.805440 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.805175    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fst9g\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-kube-api-access-fst9g\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.805440 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.805184    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:20.805440 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:20.805191    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84617ca8-7c00-4f86-99a5-5dd1b9d72329-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:21.470879 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.470838    2584 generic.go:356] "Generic (PLEG): container finished" podID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerID="5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb" exitCode=0
Jul 02 19:37:21.471060 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.470902    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"
Jul 02 19:37:21.471060 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.470918    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" event={"ID":"84617ca8-7c00-4f86-99a5-5dd1b9d72329","Type":"ContainerDied","Data":"5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb"}
Jul 02 19:37:21.471060 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.470972    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p" event={"ID":"84617ca8-7c00-4f86-99a5-5dd1b9d72329","Type":"ContainerDied","Data":"239f6563c27b40625ed7cb67af467c45b0459b5b9f16a9448859d6ddd6eba609"}
Jul 02 19:37:21.471060 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.471002    2584 scope.go:117] "RemoveContainer" containerID="5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb"
Jul 02 19:37:21.479396 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.479384    2584 scope.go:117] "RemoveContainer" containerID="20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"
Jul 02 19:37:21.486806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.486785    2584 scope.go:117] "RemoveContainer" containerID="5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb"
Jul 02 19:37:21.487100 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:37:21.487075    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb\": container with ID starting with 5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb not found: ID does not exist" containerID="5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb"
Jul 02 19:37:21.487189 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.487112    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb"} err="failed to get container status \"5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb\": rpc error: code = NotFound desc = could not find container \"5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb\": container with ID starting with 5ce47709ea12d689b36682ed478b73033db6bd163ca8ed103aef3d15934caacb not found: ID does not exist"
Jul 02 19:37:21.487189 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.487134    2584 scope.go:117] "RemoveContainer" containerID="20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"
Jul 02 19:37:21.487429 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:37:21.487408    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d\": container with ID starting with 20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d not found: ID does not exist" containerID="20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"
Jul 02 19:37:21.487505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.487436    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d"} err="failed to get container status \"20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d\": rpc error: code = NotFound desc = could not find container \"20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d\": container with ID starting with 20998688f0baa33d406b3056d666febe6084dcb7e23b56bbc600539bfa87c66d not found: ID does not exist"
Jul 02 19:37:21.487840 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.487820    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"]
Jul 02 19:37:21.492780 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:21.492760    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-fd9e49-openshift-default-5b5dff64d7-msv8p"]
Jul 02 19:37:22.273308 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:22.273273    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"]
Jul 02 19:37:22.273717 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:22.273543    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy" containerID="cri-o://605dd244d58ea894e7bfd670070a540918f0df02cde3741da96323a72bd2f890" gracePeriod=30
Jul 02 19:37:22.777007 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:22.776975    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" path="/var/lib/kubelet/pods/84617ca8-7c00-4f86-99a5-5dd1b9d72329/volumes"
Jul 02 19:37:23.174265 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174226    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"]
Jul 02 19:37:23.174704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174654    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d88c0176-8a7a-4d99-8371-33c290cbdbd1" containerName="maas-api"
Jul 02 19:37:23.174704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174675    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d88c0176-8a7a-4d99-8371-33c290cbdbd1" containerName="maas-api"
Jul 02 19:37:23.174704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174697    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy"
Jul 02 19:37:23.174704 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174706    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174716    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174724    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174743    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b235c0f0-59c9-415c-a0c9-8dce9fc51f18" containerName="maas-api"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174751    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b235c0f0-59c9-415c-a0c9-8dce9fc51f18" containerName="maas-api"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174832    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d88c0176-8a7a-4d99-8371-33c290cbdbd1" containerName="maas-api"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174847    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174857    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="84617ca8-7c00-4f86-99a5-5dd1b9d72329" containerName="istio-proxy"
Jul 02 19:37:23.175005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.174867    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b235c0f0-59c9-415c-a0c9-8dce9fc51f18" containerName="maas-api"
Jul 02 19:37:23.179851 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.179830    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.182778 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.182759    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f97d8e-openshift-default-dockercfg-kwpps\""
Jul 02 19:37:23.192920 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.192899    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"]
Jul 02 19:37:23.226359 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226320    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q876\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-kube-api-access-7q876\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226469 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226381    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-socket\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226469 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226430    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-token\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226469 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istiod-ca-cert\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226469 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226468    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-credential-socket\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226653 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226584    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-certs\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226653 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226604    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-envoy\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226719 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226660    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-podinfo\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.226753 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.226724    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-data\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.327685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327649    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-token\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.327685 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istiod-ca-cert\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-credential-socket\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327778    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-certs\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327809    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-envoy\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327834    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-podinfo\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327860    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-data\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q876\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-kube-api-access-7q876\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328171 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.327980    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-socket\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328598 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.328218    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-certs\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328598 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.328285    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-credential-socket\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328598 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.328518    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istiod-ca-cert\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328598 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.328581    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-data\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.328598 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.328594    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-socket\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.330127 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.330102    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-envoy\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.330409 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.330391    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-podinfo\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.336413 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.336333    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-token\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.336618 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.336602    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q876\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-kube-api-access-7q876\") pod \"e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") " pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.491925 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.491843    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:23.652480 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.652455    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"]
Jul 02 19:37:23.653845 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:37:23.653820    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9367ada_bac5_468c_bfae_dd50bf8a6b83.slice/crio-0c488ae33ee68cc38531b47cba52385cc61e5446fbcee5f59ce524217cd9f1a6 WatchSource:0}: Error finding container 0c488ae33ee68cc38531b47cba52385cc61e5446fbcee5f59ce524217cd9f1a6: Status 404 returned error can't find the container with id 0c488ae33ee68cc38531b47cba52385cc61e5446fbcee5f59ce524217cd9f1a6
Jul 02 19:37:23.660783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.656683    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:23.660783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.656852    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:23.660783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:23.656937    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:24.488002 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:24.487968    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" event={"ID":"f9367ada-bac5-468c-bfae-dd50bf8a6b83","Type":"ContainerStarted","Data":"29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"}
Jul 02 19:37:24.488398 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:24.488021    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" event={"ID":"f9367ada-bac5-468c-bfae-dd50bf8a6b83","Type":"ContainerStarted","Data":"0c488ae33ee68cc38531b47cba52385cc61e5446fbcee5f59ce524217cd9f1a6"}
Jul 02 19:37:24.492229 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:24.492206    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:24.497546 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:24.497527    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:24.510757 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:24.510716    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podStartSLOduration=1.510702778 podStartE2EDuration="1.510702778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:37:24.508242801 +0000 UTC m=+2024.363174571" watchObservedRunningTime="2026-07-02 19:37:24.510702778 +0000 UTC m=+2024.365634559"
Jul 02 19:37:25.492813 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:25.492781    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:25.493588 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:25.493569    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:26.802730 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.802696    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"]
Jul 02 19:37:26.806613 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.806591    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.809103 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.809084    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f97d8e-serving-cert\""
Jul 02 19:37:26.822106 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.822081    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"]
Jul 02 19:37:26.857377 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.857321    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw4lv\" (UniqueName: \"kubernetes.io/projected/42a08883-28bb-440e-8474-123b16611429-kube-api-access-cw4lv\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.857505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.857392    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a08883-28bb-440e-8474-123b16611429-maas-api-tls\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.857505 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.857414    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a08883-28bb-440e-8474-123b16611429-openshift-service-ca\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.958760 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.958720    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw4lv\" (UniqueName: \"kubernetes.io/projected/42a08883-28bb-440e-8474-123b16611429-kube-api-access-cw4lv\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.958946 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.958782    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a08883-28bb-440e-8474-123b16611429-maas-api-tls\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.958946 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.958814    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a08883-28bb-440e-8474-123b16611429-openshift-service-ca\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.959671 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.959638    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a08883-28bb-440e-8474-123b16611429-openshift-service-ca\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.964447 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.964423    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a08883-28bb-440e-8474-123b16611429-maas-api-tls\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:26.970822 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:26.970794    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw4lv\" (UniqueName: \"kubernetes.io/projected/42a08883-28bb-440e-8474-123b16611429-kube-api-access-cw4lv\") pod \"maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:27.117812 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.117780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:27.255005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.254976    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"]
Jul 02 19:37:27.256813 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:37:27.256781    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a08883_28bb_440e_8474_123b16611429.slice/crio-c854e0bcd19a5627594e8fef25875bc3b021d52cd1b1795cdcc7d5a02d0cfe8d WatchSource:0}: Error finding container c854e0bcd19a5627594e8fef25875bc3b021d52cd1b1795cdcc7d5a02d0cfe8d: Status 404 returned error can't find the container with id c854e0bcd19a5627594e8fef25875bc3b021d52cd1b1795cdcc7d5a02d0cfe8d
Jul 02 19:37:27.504490 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.504447    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk" event={"ID":"42a08883-28bb-440e-8474-123b16611429","Type":"ContainerStarted","Data":"c854e0bcd19a5627594e8fef25875bc3b021d52cd1b1795cdcc7d5a02d0cfe8d"}
Jul 02 19:37:27.506601 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.506571    2584 generic.go:356] "Generic (PLEG): container finished" podID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerID="605dd244d58ea894e7bfd670070a540918f0df02cde3741da96323a72bd2f890" exitCode=0
Jul 02 19:37:27.506713 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.506607    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" event={"ID":"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55","Type":"ContainerDied","Data":"605dd244d58ea894e7bfd670070a540918f0df02cde3741da96323a72bd2f890"}
Jul 02 19:37:27.506713 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.506664    2584 scope.go:117] "RemoveContainer" containerID="b61e76dae1347c1bf304563512aefd3337786f170b285365fef1aa16dc4c71ef"
Jul 02 19:37:27.527207 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.527187    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:37:27.565562 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565530    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istiod-ca-cert\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565628    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-credential-socket\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565664    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pzcl\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-kube-api-access-7pzcl\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565701    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-data\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565731    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-socket\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565775    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-certs\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565811    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-podinfo\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565854    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-token\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.565961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.565926    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:37:27.566167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566034    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:27.566167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566102    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:27.566167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566122    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-envoy\") pod \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\" (UID: \"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55\") "
Jul 02 19:37:27.566876 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566550    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.566876 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566575    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.566876 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566589    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.566876 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566631    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:27.566876 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.566801    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:27.568147 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.568117    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-kube-api-access-7pzcl" (OuterVolumeSpecName: "kube-api-access-7pzcl") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "kube-api-access-7pzcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:27.568250 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.568143    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:37:27.568366 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.568333    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:27.568562 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.568543    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" (UID: "c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:27.667798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.667704    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pzcl\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-kube-api-access-7pzcl\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.667798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.667731    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.667798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.667741    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.667798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.667751    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.667798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.667759    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:27.667798 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:27.667768    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:28.513399 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.513359    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk" event={"ID":"42a08883-28bb-440e-8474-123b16611429","Type":"ContainerStarted","Data":"89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47"}
Jul 02 19:37:28.513822 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.513498    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:28.514690 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.514673    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"
Jul 02 19:37:28.514783 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.514671    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n" event={"ID":"c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55","Type":"ContainerDied","Data":"e38876976697ba6c7603fe89cafeaebfd9b331fa0bbdcd048e6cf1436aeaa73a"}
Jul 02 19:37:28.514833 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.514807    2584 scope.go:117] "RemoveContainer" containerID="605dd244d58ea894e7bfd670070a540918f0df02cde3741da96323a72bd2f890"
Jul 02 19:37:28.543005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.542961    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk" podStartSLOduration=1.7717653420000001 podStartE2EDuration="2.542948298s" totalImagesPullingTime="771.182956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:37:28.529302942 +0000 UTC m=+2028.384234727" watchObservedRunningTime="2026-07-02 19:37:28.542948298 +0000 UTC m=+2028.397880068"
Jul 02 19:37:28.543194 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.543181    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"]
Jul 02 19:37:28.547464 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.547442    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-78744c-openshift-default-f9d97d7df-b2x8n"]
Jul 02 19:37:28.777389 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:28.777296    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" path="/var/lib/kubelet/pods/c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55/volumes"
Jul 02 19:37:31.950410 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950376    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"]
Jul 02 19:37:31.950848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950714    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy"
Jul 02 19:37:31.950848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950728    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy"
Jul 02 19:37:31.950848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950740    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy"
Jul 02 19:37:31.950848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950748    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy"
Jul 02 19:37:31.950848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950808    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy"
Jul 02 19:37:31.950848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.950816    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b3e2d2-fdd2-4e58-97a9-165ac1bb4e55" containerName="istio-proxy"
Jul 02 19:37:31.959467 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.954426    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:31.959897 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.959874    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4be470-openshift-default-dockercfg-rxln4\""
Jul 02 19:37:31.969570 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:31.969519    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"]
Jul 02 19:37:32.007767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/911909bc-e6c6-4185-8f79-18e27f47c769-istio-podinfo\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007767 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/911909bc-e6c6-4185-8f79-18e27f47c769-istiod-ca-cert\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-certs\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007816    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkbkv\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-kube-api-access-lkbkv\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007879    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-credential-socket\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007900    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-istio-token\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007924    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-socket\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.007952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.007941    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-data\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.008176 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.008006    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-envoy\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108600 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-envoy\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/911909bc-e6c6-4185-8f79-18e27f47c769-istio-podinfo\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/911909bc-e6c6-4185-8f79-18e27f47c769-istiod-ca-cert\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108681    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-certs\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108721    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkbkv\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-kube-api-access-lkbkv\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108753    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-credential-socket\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.108797 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108773    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-istio-token\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109125 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108808    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-socket\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109125 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.108836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-data\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109125 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.109061    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-certs\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109125 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.109112    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-credential-socket\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109426 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.109396    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-data\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.109475    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-socket\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.109563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.109492    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/911909bc-e6c6-4185-8f79-18e27f47c769-istiod-ca-cert\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.111037 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.111017    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-envoy\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.111174 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.111158    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/911909bc-e6c6-4185-8f79-18e27f47c769-istio-podinfo\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.118193 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.118150    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-istio-token\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.118441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.118418    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkbkv\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-kube-api-access-lkbkv\") pod \"e2e-api-b-4be470-openshift-default-796559845-7kg9x\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") " pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.270549 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.270466    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:32.406328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.406300    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"]
Jul 02 19:37:32.407989 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:37:32.407959    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911909bc_e6c6_4185_8f79_18e27f47c769.slice/crio-3dfabe3f95cfb965117005ea683819bb63398daccc146a27273f248b2ffc03bc WatchSource:0}: Error finding container 3dfabe3f95cfb965117005ea683819bb63398daccc146a27273f248b2ffc03bc: Status 404 returned error can't find the container with id 3dfabe3f95cfb965117005ea683819bb63398daccc146a27273f248b2ffc03bc
Jul 02 19:37:32.410304 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.410269    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:32.410450 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.410334    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:32.410450 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.410392    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:32.536566 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.536464    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" event={"ID":"911909bc-e6c6-4185-8f79-18e27f47c769","Type":"ContainerStarted","Data":"f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"}
Jul 02 19:37:32.536566 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.536528    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" event={"ID":"911909bc-e6c6-4185-8f79-18e27f47c769","Type":"ContainerStarted","Data":"3dfabe3f95cfb965117005ea683819bb63398daccc146a27273f248b2ffc03bc"}
Jul 02 19:37:32.563585 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:32.563535    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podStartSLOduration=1.5635189120000001 podStartE2EDuration="1.563518912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:37:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:37:32.56203426 +0000 UTC m=+2032.416966041" watchObservedRunningTime="2026-07-02 19:37:32.563518912 +0000 UTC m=+2032.418450681"
Jul 02 19:37:33.271610 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:33.271576    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:33.276181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:33.276152    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:33.541529 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:33.541444    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:33.542414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:33.542394    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:34.526365 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:34.526321    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:41.534584 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:41.534535    2584 patch_prober.go:28] interesting pod/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 02 19:37:41.535211 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:41.535174    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 02 19:37:42.590386 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:42.590330    2584 generic.go:356] "Generic (PLEG): container finished" podID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerID="29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98" exitCode=0
Jul 02 19:37:42.590808 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:42.590404    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" event={"ID":"f9367ada-bac5-468c-bfae-dd50bf8a6b83","Type":"ContainerDied","Data":"29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"}
Jul 02 19:37:42.590865 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:42.590814    2584 scope.go:117] "RemoveContainer" containerID="29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"
Jul 02 19:37:42.591839 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:42.591809    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:42.591959 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:42.591892    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:42.591959 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:42.591941    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:43.599751 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:43.599721    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" event={"ID":"f9367ada-bac5-468c-bfae-dd50bf8a6b83","Type":"ContainerStarted","Data":"a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3"}
Jul 02 19:37:44.492900 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:44.492868    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:44.493150 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:44.493115    2584 patch_prober.go:28] interesting pod/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 02 19:37:44.493200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:44.493179    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 02 19:37:45.492959 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:45.492921    2584 patch_prober.go:28] interesting pod/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 02 19:37:45.493368 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:45.492977    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 02 19:37:46.493081 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:46.493042    2584 patch_prober.go:28] interesting pod/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 02 19:37:46.493478 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:46.493098    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 02 19:37:47.492564 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:47.492526    2584 patch_prober.go:28] interesting pod/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 02 19:37:47.492752 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:47.492584    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 02 19:37:48.201540 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.201504    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"]
Jul 02 19:37:48.201956 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.201787    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk" podUID="42a08883-28bb-440e-8474-123b16611429" containerName="maas-api" containerID="cri-o://89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47" gracePeriod=30
Jul 02 19:37:48.496324 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.496297    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:48.496814 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.496792    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:48.497499 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.497480    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:48.511615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.511595    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:48.542625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.542595    2584 patch_prober.go:28] interesting pod/e2e-api-b-4be470-openshift-default-796559845-7kg9x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 02 19:37:48.542733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.542638    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 02 19:37:48.624626 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.624596    2584 generic.go:356] "Generic (PLEG): container finished" podID="911909bc-e6c6-4185-8f79-18e27f47c769" containerID="f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b" exitCode=0
Jul 02 19:37:48.624794 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.624664    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" event={"ID":"911909bc-e6c6-4185-8f79-18e27f47c769","Type":"ContainerDied","Data":"f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"}
Jul 02 19:37:48.625107 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.625088    2584 scope.go:117] "RemoveContainer" containerID="f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"
Jul 02 19:37:48.625872 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.625849    2584 generic.go:356] "Generic (PLEG): container finished" podID="42a08883-28bb-440e-8474-123b16611429" containerID="89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47" exitCode=0
Jul 02 19:37:48.625961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.625888    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"
Jul 02 19:37:48.625961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.625921    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk" event={"ID":"42a08883-28bb-440e-8474-123b16611429","Type":"ContainerDied","Data":"89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47"}
Jul 02 19:37:48.626045 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.625977    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk" event={"ID":"42a08883-28bb-440e-8474-123b16611429","Type":"ContainerDied","Data":"c854e0bcd19a5627594e8fef25875bc3b021d52cd1b1795cdcc7d5a02d0cfe8d"}
Jul 02 19:37:48.626045 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.625999    2584 scope.go:117] "RemoveContainer" containerID="89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47"
Jul 02 19:37:48.626218 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.626170    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:48.626315 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.626274    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:48.626378 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.626320    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:37:48.636274 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.636260    2584 scope.go:117] "RemoveContainer" containerID="89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47"
Jul 02 19:37:48.636542 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:37:48.636522    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47\": container with ID starting with 89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47 not found: ID does not exist" containerID="89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47"
Jul 02 19:37:48.636618 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.636550    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47"} err="failed to get container status \"89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47\": rpc error: code = NotFound desc = could not find container \"89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47\": container with ID starting with 89cd218d0bd0c8e743492a655d9af1da614dcf61d3ec7a951fa7e97bc8dc1e47 not found: ID does not exist"
Jul 02 19:37:48.678565 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.678543    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw4lv\" (UniqueName: \"kubernetes.io/projected/42a08883-28bb-440e-8474-123b16611429-kube-api-access-cw4lv\") pod \"42a08883-28bb-440e-8474-123b16611429\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") "
Jul 02 19:37:48.678705 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.678591    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a08883-28bb-440e-8474-123b16611429-maas-api-tls\") pod \"42a08883-28bb-440e-8474-123b16611429\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") "
Jul 02 19:37:48.678705 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.678621    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a08883-28bb-440e-8474-123b16611429-openshift-service-ca\") pod \"42a08883-28bb-440e-8474-123b16611429\" (UID: \"42a08883-28bb-440e-8474-123b16611429\") "
Jul 02 19:37:48.679069 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.679031    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a08883-28bb-440e-8474-123b16611429-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42a08883-28bb-440e-8474-123b16611429" (UID: "42a08883-28bb-440e-8474-123b16611429"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:37:48.680903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.680682    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a08883-28bb-440e-8474-123b16611429-kube-api-access-cw4lv" (OuterVolumeSpecName: "kube-api-access-cw4lv") pod "42a08883-28bb-440e-8474-123b16611429" (UID: "42a08883-28bb-440e-8474-123b16611429"). InnerVolumeSpecName "kube-api-access-cw4lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:48.680903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.680726    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a08883-28bb-440e-8474-123b16611429-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42a08883-28bb-440e-8474-123b16611429" (UID: "42a08883-28bb-440e-8474-123b16611429"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:37:48.780017 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.779924    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a08883-28bb-440e-8474-123b16611429-openshift-service-ca\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:48.780178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.780093    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw4lv\" (UniqueName: \"kubernetes.io/projected/42a08883-28bb-440e-8474-123b16611429-kube-api-access-cw4lv\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:48.780178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.780112    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a08883-28bb-440e-8474-123b16611429-maas-api-tls\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:48.943731 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.943695    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"]
Jul 02 19:37:48.946579 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:48.946552    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f97d8e-6c46d85cb6-mdfdk"]
Jul 02 19:37:49.638955 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:49.638896    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" event={"ID":"911909bc-e6c6-4185-8f79-18e27f47c769","Type":"ContainerStarted","Data":"969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c"}
Jul 02 19:37:50.271065 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:50.271023    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:50.271287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:50.271259    2584 patch_prober.go:28] interesting pod/e2e-api-b-4be470-openshift-default-796559845-7kg9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 02 19:37:50.271402 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:50.271370    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 02 19:37:50.776621 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:50.776593    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a08883-28bb-440e-8474-123b16611429" path="/var/lib/kubelet/pods/42a08883-28bb-440e-8474-123b16611429/volumes"
Jul 02 19:37:51.271761 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:51.271723    2584 patch_prober.go:28] interesting pod/e2e-api-b-4be470-openshift-default-796559845-7kg9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 02 19:37:51.271921 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:51.271777    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 02 19:37:52.271026 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:52.270988    2584 patch_prober.go:28] interesting pod/e2e-api-b-4be470-openshift-default-796559845-7kg9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 02 19:37:52.271452 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:52.271046    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 02 19:37:53.271068 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:53.271029    2584 patch_prober.go:28] interesting pod/e2e-api-b-4be470-openshift-default-796559845-7kg9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 02 19:37:53.271479 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:53.271084    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 02 19:37:53.796350 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:53.796293    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"]
Jul 02 19:37:53.796787 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:53.796589    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy" containerID="cri-o://a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3" gracePeriod=30
Jul 02 19:37:54.275295 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:54.275265    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:54.275737 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:54.275628    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:54.276269 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:54.276244    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:37:59.050656 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.050630    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:59.071938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.071870    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-socket\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.071938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.071906    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-token\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.071938 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.071927    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q876\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-kube-api-access-7q876\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.071945    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-envoy\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.071969    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istiod-ca-cert\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.071997    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-credential-socket\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072062    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:59.072243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072112    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-podinfo\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072243 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072188    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-data\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072600 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072285    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-certs\") pod \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\" (UID: \"f9367ada-bac5-468c-bfae-dd50bf8a6b83\") "
Jul 02 19:37:59.072600 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072429    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:37:59.072710 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072577    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:59.072710 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072642    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:59.072710 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072693    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.072710 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072711    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.072909 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.072721    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.073279 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.073241    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:59.074550 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.074521    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-kube-api-access-7q876" (OuterVolumeSpecName: "kube-api-access-7q876") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "kube-api-access-7q876". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:59.074641 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.074562    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:37:59.074641 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.074627    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:37:59.074834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.074813    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9367ada-bac5-468c-bfae-dd50bf8a6b83" (UID: "f9367ada-bac5-468c-bfae-dd50bf8a6b83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:37:59.173846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.173811    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.173846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.173839    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.173846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.173848    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q876\" (UniqueName: \"kubernetes.io/projected/f9367ada-bac5-468c-bfae-dd50bf8a6b83-kube-api-access-7q876\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.173846 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.173857    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.174115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.173866    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.174115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.173874    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9367ada-bac5-468c-bfae-dd50bf8a6b83-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:37:59.688561 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.688528    2584 generic.go:356] "Generic (PLEG): container finished" podID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerID="a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3" exitCode=0
Jul 02 19:37:59.688744 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.688577    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"
Jul 02 19:37:59.688744 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.688616    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" event={"ID":"f9367ada-bac5-468c-bfae-dd50bf8a6b83","Type":"ContainerDied","Data":"a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3"}
Jul 02 19:37:59.688744 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.688672    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n" event={"ID":"f9367ada-bac5-468c-bfae-dd50bf8a6b83","Type":"ContainerDied","Data":"0c488ae33ee68cc38531b47cba52385cc61e5446fbcee5f59ce524217cd9f1a6"}
Jul 02 19:37:59.688744 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.688698    2584 scope.go:117] "RemoveContainer" containerID="a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3"
Jul 02 19:37:59.697994 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.697979    2584 scope.go:117] "RemoveContainer" containerID="29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"
Jul 02 19:37:59.705903 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.705889    2584 scope.go:117] "RemoveContainer" containerID="a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3"
Jul 02 19:37:59.706139 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:37:59.706121    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3\": container with ID starting with a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3 not found: ID does not exist" containerID="a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3"
Jul 02 19:37:59.706188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.706148    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3"} err="failed to get container status \"a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3\": rpc error: code = NotFound desc = could not find container \"a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3\": container with ID starting with a31b5662435f1521de584de036d3c487b1fb77b8d25fc818b3a22b559e3a7fd3 not found: ID does not exist"
Jul 02 19:37:59.706188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.706166    2584 scope.go:117] "RemoveContainer" containerID="29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"
Jul 02 19:37:59.706415 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:37:59.706396    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98\": container with ID starting with 29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98 not found: ID does not exist" containerID="29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"
Jul 02 19:37:59.706485 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.706425    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98"} err="failed to get container status \"29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98\": rpc error: code = NotFound desc = could not find container \"29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98\": container with ID starting with 29d77fe92c522c6883ce02e320bbc86f42332c925d28025e842cc2457c05ba98 not found: ID does not exist"
Jul 02 19:37:59.710414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.710393    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"]
Jul 02 19:37:59.714368 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:37:59.714333    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f97d8e-openshift-default-7cdd9b5d65-xcx8n"]
Jul 02 19:38:00.776969 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:00.776935    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" path="/var/lib/kubelet/pods/f9367ada-bac5-468c-bfae-dd50bf8a6b83/volumes"
Jul 02 19:38:00.966072 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:00.966036    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"]
Jul 02 19:38:00.966262 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:00.966225    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy" containerID="cri-o://969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c" gracePeriod=30
Jul 02 19:38:01.843287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.843233    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"]
Jul 02 19:38:01.844516 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.844492    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a08883-28bb-440e-8474-123b16611429" containerName="maas-api"
Jul 02 19:38:01.847625 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847600    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a08883-28bb-440e-8474-123b16611429" containerName="maas-api"
Jul 02 19:38:01.847723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847632    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy"
Jul 02 19:38:01.847723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847643    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy"
Jul 02 19:38:01.847723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847689    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy"
Jul 02 19:38:01.847723 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847698    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy"
Jul 02 19:38:01.847900 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847887    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy"
Jul 02 19:38:01.847942 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847917    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9367ada-bac5-468c-bfae-dd50bf8a6b83" containerName="istio-proxy"
Jul 02 19:38:01.847942 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.847930    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a08883-28bb-440e-8474-123b16611429" containerName="maas-api"
Jul 02 19:38:01.851371 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.851335    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.853597 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.853578    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-74b95c-openshift-default-dockercfg-hgrn5\""
Jul 02 19:38:01.856552 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.856523    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"]
Jul 02 19:38:01.897851 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.897822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df377f8-f5b5-494a-bbde-e3f346d20b55-istiod-ca-cert\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898020 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.897870    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-credential-socket\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898020 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.897908    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-envoy\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898020 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.897932    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-data\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898020 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.897970    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-podinfo\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898020 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.898012    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-socket\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898239 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.898112    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldqk7\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-kube-api-access-ldqk7\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898239 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.898159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-certs\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.898239 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.898183    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-token\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999564 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-credential-socket\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999564 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999579    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-envoy\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999596    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-data\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999616    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-podinfo\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999652    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-socket\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999685    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldqk7\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-kube-api-access-ldqk7\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-certs\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-token\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:01.999834 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:01.999799    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df377f8-f5b5-494a-bbde-e3f346d20b55-istiod-ca-cert\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.000188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.000033    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-data\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.000188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.000066    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-credential-socket\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.000188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.000136    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-socket\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.000580 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.000555    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-certs\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.000678 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.000611    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df377f8-f5b5-494a-bbde-e3f346d20b55-istiod-ca-cert\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.001909 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.001890    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-envoy\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.002219 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.002198    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-podinfo\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.007844 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.007817    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-token\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.007939 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.007909    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldqk7\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-kube-api-access-ldqk7\") pod \"e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") " pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.163147 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.163040    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:02.294624 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.294581    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"]
Jul 02 19:38:02.297450 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:38:02.297416    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df377f8_f5b5_494a_bbde_e3f346d20b55.slice/crio-3d02972d05b49d0f526241937bbe20d27058f9947c232c08707e617e34b4f4e6 WatchSource:0}: Error finding container 3d02972d05b49d0f526241937bbe20d27058f9947c232c08707e617e34b4f4e6: Status 404 returned error can't find the container with id 3d02972d05b49d0f526241937bbe20d27058f9947c232c08707e617e34b4f4e6
Jul 02 19:38:02.299926 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.299887    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:02.300013 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.299959    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:02.300013 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.299987    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:02.704990 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.704956    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" event={"ID":"3df377f8-f5b5-494a-bbde-e3f346d20b55","Type":"ContainerStarted","Data":"370ac0a6108ca097964e82d9da9c0d5ac4931a7c9b0a1d5271fb4788563c75e1"}
Jul 02 19:38:02.705286 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.705014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" event={"ID":"3df377f8-f5b5-494a-bbde-e3f346d20b55","Type":"ContainerStarted","Data":"3d02972d05b49d0f526241937bbe20d27058f9947c232c08707e617e34b4f4e6"}
Jul 02 19:38:02.728271 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:02.728215    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podStartSLOduration=1.7281959329999999 podStartE2EDuration="1.728195933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:38:02.725166824 +0000 UTC m=+2062.580098596" watchObservedRunningTime="2026-07-02 19:38:02.728195933 +0000 UTC m=+2062.583127704"
Jul 02 19:38:03.164121 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:03.164083    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:03.168801 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:03.168777    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:03.710328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:03.710292    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:03.711261 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:03.711236    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:06.225408 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.225333    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:38:06.337780 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.337747    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-envoy\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.337967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.337813    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/911909bc-e6c6-4185-8f79-18e27f47c769-istiod-ca-cert\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.337967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.337846    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-istio-token\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.337967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.337884    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-credential-socket\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.337967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.337966    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkbkv\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-kube-api-access-lkbkv\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.338184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.337997    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/911909bc-e6c6-4185-8f79-18e27f47c769-istio-podinfo\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.338184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338045    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-certs\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.338184 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338069    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-socket\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.338369 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338204    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:06.338369 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338264    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-data\") pod \"911909bc-e6c6-4185-8f79-18e27f47c769\" (UID: \"911909bc-e6c6-4185-8f79-18e27f47c769\") "
Jul 02 19:38:06.338369 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338275    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/911909bc-e6c6-4185-8f79-18e27f47c769-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:38:06.338595 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338388    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:06.338595 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338538    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:06.338692 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338673    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/911909bc-e6c6-4185-8f79-18e27f47c769-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.338733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338694    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.338733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338708    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.338733 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.338721    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.339359 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.339313    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-data" (OuterVolumeSpecName: "istio-data") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:06.340233 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.340204    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-kube-api-access-lkbkv" (OuterVolumeSpecName: "kube-api-access-lkbkv") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "kube-api-access-lkbkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:38:06.340322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.340245    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/911909bc-e6c6-4185-8f79-18e27f47c769-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:38:06.340444 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.340427    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:06.340509 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.340466    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-istio-token" (OuterVolumeSpecName: "istio-token") pod "911909bc-e6c6-4185-8f79-18e27f47c769" (UID: "911909bc-e6c6-4185-8f79-18e27f47c769"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:38:06.439200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.439167    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.439200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.439194    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.439200 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.439204    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkbkv\" (UniqueName: \"kubernetes.io/projected/911909bc-e6c6-4185-8f79-18e27f47c769-kube-api-access-lkbkv\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.439521 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.439214    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/911909bc-e6c6-4185-8f79-18e27f47c769-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.439521 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.439223    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/911909bc-e6c6-4185-8f79-18e27f47c769-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:06.726621 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.726589    2584 generic.go:356] "Generic (PLEG): container finished" podID="911909bc-e6c6-4185-8f79-18e27f47c769" containerID="969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c" exitCode=0
Jul 02 19:38:06.726799 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.726640    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"
Jul 02 19:38:06.726799 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.726657    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" event={"ID":"911909bc-e6c6-4185-8f79-18e27f47c769","Type":"ContainerDied","Data":"969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c"}
Jul 02 19:38:06.726799 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.726722    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x" event={"ID":"911909bc-e6c6-4185-8f79-18e27f47c769","Type":"ContainerDied","Data":"3dfabe3f95cfb965117005ea683819bb63398daccc146a27273f248b2ffc03bc"}
Jul 02 19:38:06.726799 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.726744    2584 scope.go:117] "RemoveContainer" containerID="969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c"
Jul 02 19:38:06.735712 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.735697    2584 scope.go:117] "RemoveContainer" containerID="f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"
Jul 02 19:38:06.743078 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.743051    2584 scope.go:117] "RemoveContainer" containerID="969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c"
Jul 02 19:38:06.743320 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:38:06.743302    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c\": container with ID starting with 969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c not found: ID does not exist" containerID="969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c"
Jul 02 19:38:06.743421 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.743326    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c"} err="failed to get container status \"969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c\": rpc error: code = NotFound desc = could not find container \"969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c\": container with ID starting with 969aa0d88125bbd7796586017cbb8b5e17e82ff3fe62051e458d2d73f03aaf2c not found: ID does not exist"
Jul 02 19:38:06.743421 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.743357    2584 scope.go:117] "RemoveContainer" containerID="f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"
Jul 02 19:38:06.743598 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:38:06.743580    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b\": container with ID starting with f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b not found: ID does not exist" containerID="f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"
Jul 02 19:38:06.743637 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.743603    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b"} err="failed to get container status \"f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b\": rpc error: code = NotFound desc = could not find container \"f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b\": container with ID starting with f5e03a5a4b0555095b08363f1849655a1f1f61521aa124cd519b1545f29b609b not found: ID does not exist"
Jul 02 19:38:06.752893 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.752839    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"]
Jul 02 19:38:06.760804 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.760782    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4be470-openshift-default-796559845-7kg9x"]
Jul 02 19:38:06.776769 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:06.776744    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" path="/var/lib/kubelet/pods/911909bc-e6c6-4185-8f79-18e27f47c769/volumes"
Jul 02 19:38:09.911961 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.911933    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"]
Jul 02 19:38:09.912322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.912261    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy"
Jul 02 19:38:09.912322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.912272    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy"
Jul 02 19:38:09.912322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.912286    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy"
Jul 02 19:38:09.912322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.912291    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy"
Jul 02 19:38:09.912484 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.912366    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy"
Jul 02 19:38:09.916919 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.916894    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.919582 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.919563    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-8e566a-openshift-default-dockercfg-nlp8g\""
Jul 02 19:38:09.929203 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.929178    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"]
Jul 02 19:38:09.969689 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-socket\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.969689 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969701    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-certs\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.969928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c87de4-3b59-4204-b666-5bb878ef9cc2-istiod-ca-cert\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.969928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969787    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-credential-socket\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.969928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969844    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-envoy\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.969928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-podinfo\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.969928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.969913    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcnhw\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-kube-api-access-gcnhw\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.970178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.970017    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-data\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:09.970178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:09.970102    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-token\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071369 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071302    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-token\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071369 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071374    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-socket\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-certs\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071425    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c87de4-3b59-4204-b666-5bb878ef9cc2-istiod-ca-cert\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-credential-socket\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071479    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-envoy\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071558    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-podinfo\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcnhw\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-kube-api-access-gcnhw\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071920 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071635    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-data\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.071920 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071783    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-socket\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.072031 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.071927    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-data\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.072090 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.072075    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-credential-socket\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.072143 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.072122    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-certs\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.072266 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.072215    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c87de4-3b59-4204-b666-5bb878ef9cc2-istiod-ca-cert\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.074554 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.074507    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-envoy\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.075762 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.074739    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-podinfo\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.093086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.093047    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-token\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.093215 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.093126    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcnhw\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-kube-api-access-gcnhw\") pod \"e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") " pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.229105 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.229020    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:10.380307 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.380208    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"]
Jul 02 19:38:10.383041 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:38:10.383011    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c87de4_3b59_4204_b666_5bb878ef9cc2.slice/crio-52c80711d0ea86cc4ea48642ee91c6489d729f537742b15b3c2144332d061dff WatchSource:0}: Error finding container 52c80711d0ea86cc4ea48642ee91c6489d729f537742b15b3c2144332d061dff: Status 404 returned error can't find the container with id 52c80711d0ea86cc4ea48642ee91c6489d729f537742b15b3c2144332d061dff
Jul 02 19:38:10.385144 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.385111    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:10.385214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.385173    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:10.385214 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.385200    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:10.753123 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.753091    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" event={"ID":"49c87de4-3b59-4204-b666-5bb878ef9cc2","Type":"ContainerStarted","Data":"20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"}
Jul 02 19:38:10.753358 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:10.753180    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" event={"ID":"49c87de4-3b59-4204-b666-5bb878ef9cc2","Type":"ContainerStarted","Data":"52c80711d0ea86cc4ea48642ee91c6489d729f537742b15b3c2144332d061dff"}
Jul 02 19:38:11.230007 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:11.229979    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:11.234844 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:11.234822    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:11.278616 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:11.278563    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" podStartSLOduration=2.278546308 podStartE2EDuration="2.278546308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:38:10.803390861 +0000 UTC m=+2070.658322631" watchObservedRunningTime="2026-07-02 19:38:11.278546308 +0000 UTC m=+2071.133478078"
Jul 02 19:38:11.757864 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:11.757830    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:11.758742 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:11.758725    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:18.712071 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.712029    2584 patch_prober.go:28] interesting pod/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 02 19:38:18.712535 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.712089    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 02 19:38:18.793743 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.793707    2584 generic.go:356] "Generic (PLEG): container finished" podID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerID="370ac0a6108ca097964e82d9da9c0d5ac4931a7c9b0a1d5271fb4788563c75e1" exitCode=0
Jul 02 19:38:18.793923 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.793778    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" event={"ID":"3df377f8-f5b5-494a-bbde-e3f346d20b55","Type":"ContainerDied","Data":"370ac0a6108ca097964e82d9da9c0d5ac4931a7c9b0a1d5271fb4788563c75e1"}
Jul 02 19:38:18.794178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.794163    2584 scope.go:117] "RemoveContainer" containerID="370ac0a6108ca097964e82d9da9c0d5ac4931a7c9b0a1d5271fb4788563c75e1"
Jul 02 19:38:18.795202 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.795172    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:18.795313 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.795255    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:18.795313 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:18.795307    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:19.800615 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:19.800581    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" event={"ID":"3df377f8-f5b5-494a-bbde-e3f346d20b55","Type":"ContainerStarted","Data":"6aec5dd6bc0bed19f2f3fa527c00fde81a8544b96ca64131497e6efdf41e6e1f"}
Jul 02 19:38:20.164105 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:20.164063    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:20.164319 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:20.164195    2584 patch_prober.go:28] interesting pod/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 02 19:38:20.164319 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:20.164238    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 02 19:38:21.163665 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:21.163635    2584 patch_prober.go:28] interesting pod/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 02 19:38:21.164033 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:21.163685    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 02 19:38:22.163766 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:22.163720    2584 patch_prober.go:28] interesting pod/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 02 19:38:22.164257 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:22.163788    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 02 19:38:23.164408 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:23.164368    2584 patch_prober.go:28] interesting pod/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 02 19:38:23.164780 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:23.164429    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 02 19:38:24.167898 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.167706    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:24.174330 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.168174    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:24.174330 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.168802    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:38:24.824725 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.824688    2584 generic.go:356] "Generic (PLEG): container finished" podID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerID="20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3" exitCode=0
Jul 02 19:38:24.824917 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.824757    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" event={"ID":"49c87de4-3b59-4204-b666-5bb878ef9cc2","Type":"ContainerDied","Data":"20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"}
Jul 02 19:38:24.825299 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.825281    2584 scope.go:117] "RemoveContainer" containerID="20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"
Jul 02 19:38:24.826168 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.826143    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:24.826260 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.826204    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:24.826260 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:24.826232    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 19:38:25.834465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:25.834426    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" event={"ID":"49c87de4-3b59-4204-b666-5bb878ef9cc2","Type":"ContainerStarted","Data":"916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e"}
Jul 02 19:38:26.231084 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:26.230978    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:26.231271 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:26.231126    2584 patch_prober.go:28] interesting pod/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 02 19:38:26.231271 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:26.231173    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 02 19:38:26.758550 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:26.758512    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:27.230304 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:27.229850    2584 patch_prober.go:28] interesting pod/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 02 19:38:27.230304 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:27.229915    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 02 19:38:28.229752 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:28.229716    2584 patch_prober.go:28] interesting pod/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 02 19:38:28.230064 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:28.230032    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 02 19:38:29.230643 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:29.230152    2584 patch_prober.go:28] interesting pod/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 02 19:38:29.230643 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:29.230216    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 02 19:38:30.233563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:30.233533    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:30.234400 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:30.234382    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:38:51.856001 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.855968    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"]
Jul 02 19:38:51.856408 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.856395    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="911909bc-e6c6-4185-8f79-18e27f47c769" containerName="istio-proxy"
Jul 02 19:38:51.858128 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.858113    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:51.861673 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.861650    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"kube-root-ca.crt\""
Jul 02 19:38:51.861823 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.861726    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"auth-test-model-8e566a-kserve-self-signed-certs\""
Jul 02 19:38:51.861823 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.861747    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"openshift-service-ca.crt\""
Jul 02 19:38:51.861823 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.861726    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"default-dockercfg-fzmtc\""
Jul 02 19:38:51.874279 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.874253    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"]
Jul 02 19:38:51.968124 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.968090    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjcp2\" (UniqueName: \"kubernetes.io/projected/051eb5bb-9141-4bd1-8f45-4e59205e7515-kube-api-access-rjcp2\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:51.968287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.968140    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-model-cache\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:51.968287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.968159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-home\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:51.968287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.968209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-dshm\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:51.968287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.968266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/051eb5bb-9141-4bd1-8f45-4e59205e7515-tls-certs\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:51.968287 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:51.968284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-tmp-dir\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069419 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069383    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-home\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069419 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069426    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-dshm\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069646 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069487    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/051eb5bb-9141-4bd1-8f45-4e59205e7515-tls-certs\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069646 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-tmp-dir\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069646 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069589    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjcp2\" (UniqueName: \"kubernetes.io/projected/051eb5bb-9141-4bd1-8f45-4e59205e7515-kube-api-access-rjcp2\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069768 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069672    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-model-cache\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069827    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-home\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.069939 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069879    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-tmp-dir\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.070008 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.069957    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-model-cache\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.071801 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.071771    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-dshm\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.071940 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.071924    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/051eb5bb-9141-4bd1-8f45-4e59205e7515-tls-certs\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.077885 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.077864    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjcp2\" (UniqueName: \"kubernetes.io/projected/051eb5bb-9141-4bd1-8f45-4e59205e7515-kube-api-access-rjcp2\") pod \"auth-test-model-8e566a-kserve-956bb8c47-9tpxx\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") " pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.168973 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.168885    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:38:52.303928 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.303904    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"]
Jul 02 19:38:52.306070 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:38:52.306041    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod051eb5bb_9141_4bd1_8f45_4e59205e7515.slice/crio-07e46ce8b869b43e338fbc58907cbe90b54ab61b6da2c90d8cb73d54dc62c836 WatchSource:0}: Error finding container 07e46ce8b869b43e338fbc58907cbe90b54ab61b6da2c90d8cb73d54dc62c836: Status 404 returned error can't find the container with id 07e46ce8b869b43e338fbc58907cbe90b54ab61b6da2c90d8cb73d54dc62c836
Jul 02 19:38:52.970684 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.970644    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" event={"ID":"051eb5bb-9141-4bd1-8f45-4e59205e7515","Type":"ContainerStarted","Data":"5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9"}
Jul 02 19:38:52.970684 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.970687    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" event={"ID":"051eb5bb-9141-4bd1-8f45-4e59205e7515","Type":"ContainerStarted","Data":"07e46ce8b869b43e338fbc58907cbe90b54ab61b6da2c90d8cb73d54dc62c836"}
Jul 02 19:38:52.995414 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:38:52.995367    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" podStartSLOduration=1.995329669 podStartE2EDuration="1.995329669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:38:52.99428037 +0000 UTC m=+2112.849212163" watchObservedRunningTime="2026-07-02 19:38:52.995329669 +0000 UTC m=+2112.850261439"
Jul 02 19:39:02.169289 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:39:02.169245    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:39:02.169710 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:39:02.169302    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:39:02.181801 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:39:02.181777    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:39:03.025616 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:39:03.025585    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:40:47.827190 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:40:47.827160    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"]
Jul 02 19:41:02.849441 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:02.849332    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" podUID="051eb5bb-9141-4bd1-8f45-4e59205e7515" containerName="main" containerID="cri-o://5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9" gracePeriod=45
Jul 02 19:41:03.016053 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.016016    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" podUID="051eb5bb-9141-4bd1-8f45-4e59205e7515" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": dial tcp 10.134.0.57:8000: connect: connection refused"
Jul 02 19:41:03.109140 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.109088    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:41:03.159893 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.159866    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjcp2\" (UniqueName: \"kubernetes.io/projected/051eb5bb-9141-4bd1-8f45-4e59205e7515-kube-api-access-rjcp2\") pod \"051eb5bb-9141-4bd1-8f45-4e59205e7515\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") "
Jul 02 19:41:03.160094 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.159907    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-home\") pod \"051eb5bb-9141-4bd1-8f45-4e59205e7515\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") "
Jul 02 19:41:03.160094 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.159960    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-tmp-dir\") pod \"051eb5bb-9141-4bd1-8f45-4e59205e7515\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") "
Jul 02 19:41:03.160094 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160013    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-model-cache\") pod \"051eb5bb-9141-4bd1-8f45-4e59205e7515\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") "
Jul 02 19:41:03.160094 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160044    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/051eb5bb-9141-4bd1-8f45-4e59205e7515-tls-certs\") pod \"051eb5bb-9141-4bd1-8f45-4e59205e7515\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") "
Jul 02 19:41:03.160094 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160090    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-dshm\") pod \"051eb5bb-9141-4bd1-8f45-4e59205e7515\" (UID: \"051eb5bb-9141-4bd1-8f45-4e59205e7515\") "
Jul 02 19:41:03.160368 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160245    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "051eb5bb-9141-4bd1-8f45-4e59205e7515" (UID: "051eb5bb-9141-4bd1-8f45-4e59205e7515"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:41:03.160368 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160256    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-home" (OuterVolumeSpecName: "home") pod "051eb5bb-9141-4bd1-8f45-4e59205e7515" (UID: "051eb5bb-9141-4bd1-8f45-4e59205e7515"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:41:03.160368 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160291    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-model-cache" (OuterVolumeSpecName: "model-cache") pod "051eb5bb-9141-4bd1-8f45-4e59205e7515" (UID: "051eb5bb-9141-4bd1-8f45-4e59205e7515"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:41:03.160486 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160423    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-home\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:41:03.160486 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160439    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-tmp-dir\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:41:03.160486 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.160448    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-model-cache\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:41:03.162221 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.162193    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-dshm" (OuterVolumeSpecName: "dshm") pod "051eb5bb-9141-4bd1-8f45-4e59205e7515" (UID: "051eb5bb-9141-4bd1-8f45-4e59205e7515"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:41:03.162328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.162253    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/051eb5bb-9141-4bd1-8f45-4e59205e7515-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "051eb5bb-9141-4bd1-8f45-4e59205e7515" (UID: "051eb5bb-9141-4bd1-8f45-4e59205e7515"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:41:03.162328 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.162279    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/051eb5bb-9141-4bd1-8f45-4e59205e7515-kube-api-access-rjcp2" (OuterVolumeSpecName: "kube-api-access-rjcp2") pod "051eb5bb-9141-4bd1-8f45-4e59205e7515" (UID: "051eb5bb-9141-4bd1-8f45-4e59205e7515"). InnerVolumeSpecName "kube-api-access-rjcp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:41:03.261786 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.261756    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/051eb5bb-9141-4bd1-8f45-4e59205e7515-tls-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:41:03.261786 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.261780    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/051eb5bb-9141-4bd1-8f45-4e59205e7515-dshm\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:41:03.261786 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.261789    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjcp2\" (UniqueName: \"kubernetes.io/projected/051eb5bb-9141-4bd1-8f45-4e59205e7515-kube-api-access-rjcp2\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:41:03.548880 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.548781    2584 generic.go:356] "Generic (PLEG): container finished" podID="051eb5bb-9141-4bd1-8f45-4e59205e7515" containerID="5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9" exitCode=0
Jul 02 19:41:03.548880 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.548829    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"
Jul 02 19:41:03.549118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.548867    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" event={"ID":"051eb5bb-9141-4bd1-8f45-4e59205e7515","Type":"ContainerDied","Data":"5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9"}
Jul 02 19:41:03.549118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.548922    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx" event={"ID":"051eb5bb-9141-4bd1-8f45-4e59205e7515","Type":"ContainerDied","Data":"07e46ce8b869b43e338fbc58907cbe90b54ab61b6da2c90d8cb73d54dc62c836"}
Jul 02 19:41:03.549118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.548945    2584 scope.go:117] "RemoveContainer" containerID="5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9"
Jul 02 19:41:03.557632 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.557613    2584 scope.go:117] "RemoveContainer" containerID="5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9"
Jul 02 19:41:03.557899 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:41:03.557881    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9\": container with ID starting with 5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9 not found: ID does not exist" containerID="5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9"
Jul 02 19:41:03.557967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.557912    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9"} err="failed to get container status \"5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9\": rpc error: code = NotFound desc = could not find container \"5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9\": container with ID starting with 5c25369310673a358185f6d43dcf2582088171b616c9e52ce3b47c37572db8d9 not found: ID does not exist"
Jul 02 19:41:03.569842 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.569818    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"]
Jul 02 19:41:03.574122 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:03.574103    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/auth-test-model-8e566a-kserve-956bb8c47-9tpxx"]
Jul 02 19:41:04.777195 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:41:04.777158    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="051eb5bb-9141-4bd1-8f45-4e59205e7515" path="/var/lib/kubelet/pods/051eb5bb-9141-4bd1-8f45-4e59205e7515/volumes"
Jul 02 19:43:01.735377 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.735323    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"]
Jul 02 19:43:01.735820 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.735714    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="051eb5bb-9141-4bd1-8f45-4e59205e7515" containerName="main"
Jul 02 19:43:01.735820 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.735726    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="051eb5bb-9141-4bd1-8f45-4e59205e7515" containerName="main"
Jul 02 19:43:01.735820 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.735801    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="051eb5bb-9141-4bd1-8f45-4e59205e7515" containerName="main"
Jul 02 19:43:01.738659 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.738641    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.742185 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.742161    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"kube-root-ca.crt\""
Jul 02 19:43:01.742311 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.742161    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"openshift-service-ca.crt\""
Jul 02 19:43:01.742311 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.742212    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"default-dockercfg-fzmtc\""
Jul 02 19:43:01.742311 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.742297    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e566a\"/\"rate-test-model-8e566a-kserve-self-signed-certs\""
Jul 02 19:43:01.748732 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.748713    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"]
Jul 02 19:43:01.807716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.807684    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tmp-dir\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.807716 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.807716    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-model-cache\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.807967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.807764    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-home\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.807967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.807797    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tls-certs\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.807967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.807815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs7v4\" (UniqueName: \"kubernetes.io/projected/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-kube-api-access-bs7v4\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.807967 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.807885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-dshm\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909100    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-home\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tls-certs\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909236    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs7v4\" (UniqueName: \"kubernetes.io/projected/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-kube-api-access-bs7v4\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-dshm\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tmp-dir\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-model-cache\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.909920    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-model-cache\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.911375 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.910393    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-home\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.914565 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.911955    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tmp-dir\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.914565 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.914445    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tls-certs\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.915722 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.915678    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-dshm\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:01.919466 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:01.919442    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs7v4\" (UniqueName: \"kubernetes.io/projected/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-kube-api-access-bs7v4\") pod \"rate-test-model-8e566a-kserve-547d5f488c-mjjvr\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") " pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:02.049089 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:02.048997    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:02.197028 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:02.196999    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"]
Jul 02 19:43:02.198824 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:43:02.198782    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd86ac9e8_fe28_49f1_8fd0_08adef33ddf1.slice/crio-4162347058f884926877b2f32c8c840058a69a309a5b196902bf8cc75ad5fbce WatchSource:0}: Error finding container 4162347058f884926877b2f32c8c840058a69a309a5b196902bf8cc75ad5fbce: Status 404 returned error can't find the container with id 4162347058f884926877b2f32c8c840058a69a309a5b196902bf8cc75ad5fbce
Jul 02 19:43:03.086841 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:03.086802    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr" event={"ID":"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1","Type":"ContainerStarted","Data":"ea8e3171523d1febbf92ab8160e2f4948f6553da0ef509479003e6bc1bb906c0"}
Jul 02 19:43:03.087242 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:03.086847    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr" event={"ID":"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1","Type":"ContainerStarted","Data":"4162347058f884926877b2f32c8c840058a69a309a5b196902bf8cc75ad5fbce"}
Jul 02 19:43:03.106982 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:03.106936    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr" podStartSLOduration=2.106921452 podStartE2EDuration="2.106921452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:43:03.104723877 +0000 UTC m=+2362.959655648" watchObservedRunningTime="2026-07-02 19:43:03.106921452 +0000 UTC m=+2362.961853222"
Jul 02 19:43:12.049976 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:12.049936    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:12.050420 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:12.050150    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:12.062385 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:12.062358    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:12.137073 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:12.137046    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:42.164205 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:42.164171    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"]
Jul 02 19:43:57.183676 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.183558    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr" podUID="d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" containerName="main" containerID="cri-o://ea8e3171523d1febbf92ab8160e2f4948f6553da0ef509479003e6bc1bb906c0" gracePeriod=45
Jul 02 19:43:57.333758 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.333719    2584 generic.go:356] "Generic (PLEG): container finished" podID="d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" containerID="ea8e3171523d1febbf92ab8160e2f4948f6553da0ef509479003e6bc1bb906c0" exitCode=0
Jul 02 19:43:57.333936 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.333783    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr" event={"ID":"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1","Type":"ContainerDied","Data":"ea8e3171523d1febbf92ab8160e2f4948f6553da0ef509479003e6bc1bb906c0"}
Jul 02 19:43:57.428927 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.428902    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:57.626093 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626062    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tls-certs\") pod \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") "
Jul 02 19:43:57.626277 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626117    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-model-cache\") pod \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") "
Jul 02 19:43:57.626277 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626154    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-dshm\") pod \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") "
Jul 02 19:43:57.626277 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626211    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tmp-dir\") pod \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") "
Jul 02 19:43:57.626277 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626253    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-home\") pod \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") "
Jul 02 19:43:57.626277 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626277    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs7v4\" (UniqueName: \"kubernetes.io/projected/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-kube-api-access-bs7v4\") pod \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\" (UID: \"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1\") "
Jul 02 19:43:57.626577 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626479    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-model-cache" (OuterVolumeSpecName: "model-cache") pod "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" (UID: "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:43:57.626577 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626567    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-model-cache\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:43:57.626648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626593    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" (UID: "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:43:57.626648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.626610    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-home" (OuterVolumeSpecName: "home") pod "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" (UID: "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:43:57.628357 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.628316    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-dshm" (OuterVolumeSpecName: "dshm") pod "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" (UID: "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:43:57.628465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.628320    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-kube-api-access-bs7v4" (OuterVolumeSpecName: "kube-api-access-bs7v4") pod "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" (UID: "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1"). InnerVolumeSpecName "kube-api-access-bs7v4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:43:57.628465 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.628362    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" (UID: "d86ac9e8-fe28-49f1-8fd0-08adef33ddf1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:43:57.727635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.727601    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-dshm\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:43:57.727635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.727631    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tmp-dir\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:43:57.727635 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.727641    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-home\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:43:57.727882 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.727649    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs7v4\" (UniqueName: \"kubernetes.io/projected/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-kube-api-access-bs7v4\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:43:57.727882 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:57.727660    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1-tls-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:43:58.340884 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:58.340855    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"
Jul 02 19:43:58.340884 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:58.340880    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr" event={"ID":"d86ac9e8-fe28-49f1-8fd0-08adef33ddf1","Type":"ContainerDied","Data":"4162347058f884926877b2f32c8c840058a69a309a5b196902bf8cc75ad5fbce"}
Jul 02 19:43:58.341456 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:58.340924    2584 scope.go:117] "RemoveContainer" containerID="ea8e3171523d1febbf92ab8160e2f4948f6553da0ef509479003e6bc1bb906c0"
Jul 02 19:43:58.362405 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:58.362361    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"]
Jul 02 19:43:58.367091 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:58.367067    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8e566a/rate-test-model-8e566a-kserve-547d5f488c-mjjvr"]
Jul 02 19:43:58.777316 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:43:58.777231    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" path="/var/lib/kubelet/pods/d86ac9e8-fe28-49f1-8fd0-08adef33ddf1/volumes"
Jul 02 19:44:26.351838 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:26.351801    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"]
Jul 02 19:44:26.352664 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:26.352043    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy" containerID="cri-o://6aec5dd6bc0bed19f2f3fa527c00fde81a8544b96ca64131497e6efdf41e6e1f" gracePeriod=30
Jul 02 19:44:31.510032 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.509991    2584 generic.go:356] "Generic (PLEG): container finished" podID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerID="6aec5dd6bc0bed19f2f3fa527c00fde81a8544b96ca64131497e6efdf41e6e1f" exitCode=0
Jul 02 19:44:31.510494 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.510053    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" event={"ID":"3df377f8-f5b5-494a-bbde-e3f346d20b55","Type":"ContainerDied","Data":"6aec5dd6bc0bed19f2f3fa527c00fde81a8544b96ca64131497e6efdf41e6e1f"}
Jul 02 19:44:31.510494 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.510168    2584 scope.go:117] "RemoveContainer" containerID="370ac0a6108ca097964e82d9da9c0d5ac4931a7c9b0a1d5271fb4788563c75e1"
Jul 02 19:44:31.613983 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.613920    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:44:31.741563 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741524    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-certs\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.741774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741577    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df377f8-f5b5-494a-bbde-e3f346d20b55-istiod-ca-cert\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.741774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741611    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldqk7\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-kube-api-access-ldqk7\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.741774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741643    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-podinfo\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.741774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741666    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-socket\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.741774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741713    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-data\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.741774 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741752    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-credential-socket\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.742115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741828    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-token\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.742115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.741860    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-envoy\") pod \"3df377f8-f5b5-494a-bbde-e3f346d20b55\" (UID: \"3df377f8-f5b5-494a-bbde-e3f346d20b55\") "
Jul 02 19:44:31.742115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742049    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:31.742115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742054    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df377f8-f5b5-494a-bbde-e3f346d20b55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:44:31.742115 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742029    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:31.742390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742181    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df377f8-f5b5-494a-bbde-e3f346d20b55-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.742390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742200    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.742390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742214    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.742390 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742285    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:31.742670 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.742647    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-data" (OuterVolumeSpecName: "istio-data") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:31.743936 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.743910    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:44:31.744086 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.743963    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-token" (OuterVolumeSpecName: "istio-token") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:44:31.744231 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.744206    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-kube-api-access-ldqk7" (OuterVolumeSpecName: "kube-api-access-ldqk7") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "kube-api-access-ldqk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:44:31.744327 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.744267    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3df377f8-f5b5-494a-bbde-e3f346d20b55" (UID: "3df377f8-f5b5-494a-bbde-e3f346d20b55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:31.842854 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.842815    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.842854 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.842846    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.842854 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.842856    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.843178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.842868    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df377f8-f5b5-494a-bbde-e3f346d20b55-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.843178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.842886    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldqk7\" (UniqueName: \"kubernetes.io/projected/3df377f8-f5b5-494a-bbde-e3f346d20b55-kube-api-access-ldqk7\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:31.843178 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:31.842895    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df377f8-f5b5-494a-bbde-e3f346d20b55-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:32.516568 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:32.516528    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh" event={"ID":"3df377f8-f5b5-494a-bbde-e3f346d20b55","Type":"ContainerDied","Data":"3d02972d05b49d0f526241937bbe20d27058f9947c232c08707e617e34b4f4e6"}
Jul 02 19:44:32.517005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:32.516596    2584 scope.go:117] "RemoveContainer" containerID="6aec5dd6bc0bed19f2f3fa527c00fde81a8544b96ca64131497e6efdf41e6e1f"
Jul 02 19:44:32.517005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:32.516607    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"
Jul 02 19:44:32.540643 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:32.540617    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"]
Jul 02 19:44:32.545268 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:32.545245    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-74b95c-openshift-default-78cb948699-jzdfh"]
Jul 02 19:44:32.777718 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:32.777628    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" path="/var/lib/kubelet/pods/3df377f8-f5b5-494a-bbde-e3f346d20b55/volumes"
Jul 02 19:44:33.509905 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:33.509868    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"]
Jul 02 19:44:33.510167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:33.510119    2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy" containerID="cri-o://916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e" gracePeriod=30
Jul 02 19:44:38.759718 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.759694    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:44:38.790426 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790358    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-certs\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790426 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790392    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-data\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790426 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790409    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-socket\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790450    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-podinfo\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790494    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-token\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790530    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c87de4-3b59-4204-b666-5bb878ef9cc2-istiod-ca-cert\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790554    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-envoy\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790599    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcnhw\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-kube-api-access-gcnhw\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790662 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790654    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-credential-socket\") pod \"49c87de4-3b59-4204-b666-5bb878ef9cc2\" (UID: \"49c87de4-3b59-4204-b666-5bb878ef9cc2\") "
Jul 02 19:44:38.790989 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790718    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:38.790989 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.790814    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:38.791097 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.791051    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-certs\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.791097 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.791071    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-workload-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.791400 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.791256    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:38.791400 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.791272    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c87de4-3b59-4204-b666-5bb878ef9cc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:44:38.791848 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.791826    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:38.792923 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.792895    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:44:38.793218 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.793191    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:44:38.793306 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.793226    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:38.793378 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.793361    2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-kube-api-access-gcnhw" (OuterVolumeSpecName: "kube-api-access-gcnhw") pod "49c87de4-3b59-4204-b666-5bb878ef9cc2" (UID: "49c87de4-3b59-4204-b666-5bb878ef9cc2"). InnerVolumeSpecName "kube-api-access-gcnhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:44:38.891880 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891846    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-credential-socket\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.891880 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891873    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-data\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.891880 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891884    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-podinfo\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.892118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891894    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-token\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.892118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891903    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c87de4-3b59-4204-b666-5bb878ef9cc2-istiod-ca-cert\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.892118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891910    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c87de4-3b59-4204-b666-5bb878ef9cc2-istio-envoy\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:38.892118 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:38.891918    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcnhw\" (UniqueName: \"kubernetes.io/projected/49c87de4-3b59-4204-b666-5bb878ef9cc2-kube-api-access-gcnhw\") on node \"ip-10-0-135-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:39.026690 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.026656    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6464bbbddb-g2vth_4ce4beab-3cfb-4db3-847a-ad22302b6469/manager/0.log"
Jul 02 19:44:39.140503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.140475    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r4hsl_9fda2112-24d7-42c5-a887-74e060e85ae4/server/0.log"
Jul 02 19:44:39.260041 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.260013    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8hgmc_cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323/manager/0.log"
Jul 02 19:44:39.383188 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.383161    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb89c69-4rjpk_8954e82e-6c64-4c92-814d-702dc9889237/copy-manifests/0.log"
Jul 02 19:44:39.390352 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.390325    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb89c69-4rjpk_8954e82e-6c64-4c92-814d-702dc9889237/manager/0.log"
Jul 02 19:44:39.549249 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.549148    2584 generic.go:356] "Generic (PLEG): container finished" podID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerID="916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e" exitCode=0
Jul 02 19:44:39.549249 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.549195    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"
Jul 02 19:44:39.549249 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.549198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" event={"ID":"49c87de4-3b59-4204-b666-5bb878ef9cc2","Type":"ContainerDied","Data":"916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e"}
Jul 02 19:44:39.549535 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.549253    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm" event={"ID":"49c87de4-3b59-4204-b666-5bb878ef9cc2","Type":"ContainerDied","Data":"52c80711d0ea86cc4ea48642ee91c6489d729f537742b15b3c2144332d061dff"}
Jul 02 19:44:39.549535 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.549280    2584 scope.go:117] "RemoveContainer" containerID="916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e"
Jul 02 19:44:39.558147 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.558131    2584 scope.go:117] "RemoveContainer" containerID="20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"
Jul 02 19:44:39.566167 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.566149    2584 scope.go:117] "RemoveContainer" containerID="916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e"
Jul 02 19:44:39.566481 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:44:39.566460    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e\": container with ID starting with 916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e not found: ID does not exist" containerID="916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e"
Jul 02 19:44:39.566538 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.566493    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e"} err="failed to get container status \"916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e\": rpc error: code = NotFound desc = could not find container \"916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e\": container with ID starting with 916ba53421e10569f714b9a5b070ae36233c50d1d7492e25052058b8bbb6811e not found: ID does not exist"
Jul 02 19:44:39.566538 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.566512    2584 scope.go:117] "RemoveContainer" containerID="20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"
Jul 02 19:44:39.566761 ip-10-0-135-120 kubenswrapper[2584]: E0702 19:44:39.566743    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3\": container with ID starting with 20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3 not found: ID does not exist" containerID="20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"
Jul 02 19:44:39.566806 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.566768    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3"} err="failed to get container status \"20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3\": rpc error: code = NotFound desc = could not find container \"20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3\": container with ID starting with 20c91d4b20537d5b0ae5b48bc912d177ed422fe987104294a95ef0aa9d96bfd3 not found: ID does not exist"
Jul 02 19:44:39.570627 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.570605    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"]
Jul 02 19:44:39.576947 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:39.576927    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-8e566a-openshift-default-5774b64c5f-fl9mm"]
Jul 02 19:44:40.778093 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:40.778057    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" path="/var/lib/kubelet/pods/49c87de4-3b59-4204-b666-5bb878ef9cc2/volumes"
Jul 02 19:44:42.145011 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:42.144983    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb67864b99_cedd2fc4-ac14-437d-94dd-61a3fc625a1a/istio-proxy/0.log"
Jul 02 19:44:42.260413 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:42.260384    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ks5b4_a87ccab5-02ad-45f9-93b2-6681039a7b99/discovery/0.log"
Jul 02 19:44:42.481552 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:42.481471    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-578747f885-tgqz8_88504682-7871-4c59-95cb-cb412de7f35c/kube-auth-proxy/0.log"
Jul 02 19:44:42.609005 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:42.608975    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/1.log"
Jul 02 19:44:42.843849 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:42.843816    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ddfm5_78211ab9-79b8-43d1-902a-fd8b7cb524e0/payload-processing/0.log"
Jul 02 19:44:43.302382 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:43.302286    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dmgg2_6da319c4-0364-48a6-a6c2-082b9bf029da/main/0.log"
Jul 02 19:44:43.413592 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:43.413545    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jpk2z_53c7546d-602a-4336-85e9-0ba992e1fc48/main/0.log"
Jul 02 19:44:51.440787 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:51.440754    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hfqzn_631168e3-18d4-4ec6-9eb9-aea35e3f50d1/global-pull-secret-syncer/0.log"
Jul 02 19:44:51.614830 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:51.614801    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d5vf8_a5b75400-87e5-4124-a321-4852c2d15aec/konnectivity-agent/0.log"
Jul 02 19:44:51.696154 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:51.696072    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-120.ec2.internal_6f49a5dc7fed05757f7e38b0fbf5bfb5/haproxy/0.log"
Jul 02 19:44:59.778906 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:59.778866    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-f2dkn_b98df59c-b29e-4255-85c1-5bcdfe556aa6/monitoring-plugin/0.log"
Jul 02 19:44:59.910845 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:59.910804    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r4lv9_b6d210e0-c015-494e-bcd4-92d75be978e9/node-exporter/0.log"
Jul 02 19:44:59.937701 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:59.937669    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r4lv9_b6d210e0-c015-494e-bcd4-92d75be978e9/kube-rbac-proxy/0.log"
Jul 02 19:44:59.963273 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:44:59.963246    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r4lv9_b6d210e0-c015-494e-bcd4-92d75be978e9/init-textfile/0.log"
Jul 02 19:45:00.145977 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.145944    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"]
Jul 02 19:45:00.146278 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146266    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy"
Jul 02 19:45:00.146322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146280    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy"
Jul 02 19:45:00.146322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146291    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy"
Jul 02 19:45:00.146322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146298    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy"
Jul 02 19:45:00.146322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146313    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" containerName="main"
Jul 02 19:45:00.146322 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146319    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" containerName="main"
Jul 02 19:45:00.146503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146392    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy"
Jul 02 19:45:00.146503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146400    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d86ac9e8-fe28-49f1-8fd0-08adef33ddf1" containerName="main"
Jul 02 19:45:00.146503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146406    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy"
Jul 02 19:45:00.146503 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.146413    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy"
Jul 02 19:45:00.149315 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.149298    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"
Jul 02 19:45:00.158952 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.158924    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"]
Jul 02 19:45:00.282977 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.282944    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqts\" (UniqueName: \"kubernetes.io/projected/b998913c-fd0e-41ef-9da4-4dc301c9d669-kube-api-access-kxqts\") pod \"maas-api-key-cleanup-29717025-dznn2\" (UID: \"b998913c-fd0e-41ef-9da4-4dc301c9d669\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"
Jul 02 19:45:00.383937 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.383900    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqts\" (UniqueName: \"kubernetes.io/projected/b998913c-fd0e-41ef-9da4-4dc301c9d669-kube-api-access-kxqts\") pod \"maas-api-key-cleanup-29717025-dznn2\" (UID: \"b998913c-fd0e-41ef-9da4-4dc301c9d669\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"
Jul 02 19:45:00.392738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.392703    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqts\" (UniqueName: \"kubernetes.io/projected/b998913c-fd0e-41ef-9da4-4dc301c9d669-kube-api-access-kxqts\") pod \"maas-api-key-cleanup-29717025-dznn2\" (UID: \"b998913c-fd0e-41ef-9da4-4dc301c9d669\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"
Jul 02 19:45:00.459826 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.459737    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"
Jul 02 19:45:00.616487 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.616455    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2"]
Jul 02 19:45:00.618744 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:45:00.618715    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb998913c_fd0e_41ef_9da4_4dc301c9d669.slice/crio-990513ea8e4ff5b3cb3d2cd2879d1dadf210104265f9153b03a48f51bd264edb WatchSource:0}: Error finding container 990513ea8e4ff5b3cb3d2cd2879d1dadf210104265f9153b03a48f51bd264edb: Status 404 returned error can't find the container with id 990513ea8e4ff5b3cb3d2cd2879d1dadf210104265f9153b03a48f51bd264edb
Jul 02 19:45:00.621141 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.621123    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:45:00.644866 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:00.644831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2" event={"ID":"b998913c-fd0e-41ef-9da4-4dc301c9d669","Type":"ContainerStarted","Data":"990513ea8e4ff5b3cb3d2cd2879d1dadf210104265f9153b03a48f51bd264edb"}
Jul 02 19:45:01.923469 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.923434    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"]
Jul 02 19:45:01.924181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.923941    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy"
Jul 02 19:45:01.924181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.923964    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy"
Jul 02 19:45:01.924181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.923980    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy"
Jul 02 19:45:01.924181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.923987    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c87de4-3b59-4204-b666-5bb878ef9cc2" containerName="istio-proxy"
Jul 02 19:45:01.924181 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.924084    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3df377f8-f5b5-494a-bbde-e3f346d20b55" containerName="istio-proxy"
Jul 02 19:45:01.927439 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.927417    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:01.929905 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.929882    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-88s92\"/\"openshift-service-ca.crt\""
Jul 02 19:45:01.930514 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.930495    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-88s92\"/\"default-dockercfg-ll29f\""
Jul 02 19:45:01.930634 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.930541    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-88s92\"/\"kube-root-ca.crt\""
Jul 02 19:45:01.938211 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:01.938140    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"]
Jul 02 19:45:02.000831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.000791    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-proc\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.000831 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.000835    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-lib-modules\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.001070 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.000903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-sys\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.001070 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.000939    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-podres\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.001070 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.000965    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nzsm\" (UniqueName: \"kubernetes.io/projected/901caea3-e7c5-4b1e-9706-b641cc697ebc-kube-api-access-9nzsm\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.101948 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.101915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-sys\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.101965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-podres\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.101997    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nzsm\" (UniqueName: \"kubernetes.io/projected/901caea3-e7c5-4b1e-9706-b641cc697ebc-kube-api-access-9nzsm\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.102059    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-sys\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.102071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-proc\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.102112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-lib-modules\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.102129    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-podres\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102156 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.102117    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-proc\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.102574 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.102210    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/901caea3-e7c5-4b1e-9706-b641cc697ebc-lib-modules\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.110711 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.110677    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nzsm\" (UniqueName: \"kubernetes.io/projected/901caea3-e7c5-4b1e-9706-b641cc697ebc-kube-api-access-9nzsm\") pod \"perf-node-gather-daemonset-5q6md\" (UID: \"901caea3-e7c5-4b1e-9706-b641cc697ebc\") " pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.241633 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.241532    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:02.973904 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:02.973876    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"]
Jul 02 19:45:02.976403 ip-10-0-135-120 kubenswrapper[2584]: W0702 19:45:02.976376    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod901caea3_e7c5_4b1e_9706_b641cc697ebc.slice/crio-d5602b3b9d7152117ef59050f6cd424ddad04b76433a8cb5d18dd2933204a9a9 WatchSource:0}: Error finding container d5602b3b9d7152117ef59050f6cd424ddad04b76433a8cb5d18dd2933204a9a9: Status 404 returned error can't find the container with id d5602b3b9d7152117ef59050f6cd424ddad04b76433a8cb5d18dd2933204a9a9
Jul 02 19:45:03.660539 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:03.660498    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2" event={"ID":"b998913c-fd0e-41ef-9da4-4dc301c9d669","Type":"ContainerStarted","Data":"4faa78f5df14aebaf8feab269c45b79a00a7ebef7fb8d55b13630406e389cbf3"}
Jul 02 19:45:03.662061 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:03.662031    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md" event={"ID":"901caea3-e7c5-4b1e-9706-b641cc697ebc","Type":"ContainerStarted","Data":"3037a22d7f47eba9054add3474e0656e3e2f3db0c36f30a6182869f4f4c33061"}
Jul 02 19:45:03.662204 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:03.662079    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md" event={"ID":"901caea3-e7c5-4b1e-9706-b641cc697ebc","Type":"ContainerStarted","Data":"d5602b3b9d7152117ef59050f6cd424ddad04b76433a8cb5d18dd2933204a9a9"}
Jul 02 19:45:03.662204 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:03.662156    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:03.675995 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:03.675937    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29717025-dznn2" podStartSLOduration=1.391766775 podStartE2EDuration="3.67590981s" totalImagesPullingTime="2.284143035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:45:03.674694648 +0000 UTC m=+2483.529626417" watchObservedRunningTime="2026-07-02 19:45:03.67590981 +0000 UTC m=+2483.530841581"
Jul 02 19:45:03.693648 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:03.693604    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md" podStartSLOduration=2.693588868 podStartE2EDuration="2.693588868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:45:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:45:03.691957929 +0000 UTC m=+2483.546889701" watchObservedRunningTime="2026-07-02 19:45:03.693588868 +0000 UTC m=+2483.548520638"
Jul 02 19:45:04.784446 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:04.784417    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fzfd4_26d60aa7-42cc-4c6f-bfd4-8634586894b4/dns/0.log"
Jul 02 19:45:04.809845 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:04.809818    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fzfd4_26d60aa7-42cc-4c6f-bfd4-8634586894b4/kube-rbac-proxy/0.log"
Jul 02 19:45:04.957134 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:04.957097    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9sz77_6079c29a-a61e-4487-96c2-831e5f272a17/dns-node-resolver/0.log"
Jul 02 19:45:05.579690 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:05.579655    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sxpt2_f5963213-fcb7-4e5d-a018-152633bf39e8/node-ca/0.log"
Jul 02 19:45:06.581027 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:06.580998    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb67864b99_cedd2fc4-ac14-437d-94dd-61a3fc625a1a/istio-proxy/0.log"
Jul 02 19:45:06.695908 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:06.695877    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ks5b4_a87ccab5-02ad-45f9-93b2-6681039a7b99/discovery/0.log"
Jul 02 19:45:06.743829 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:06.743801    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-578747f885-tgqz8_88504682-7871-4c59-95cb-cb412de7f35c/kube-auth-proxy/0.log"
Jul 02 19:45:06.772684 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:06.772656    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/0.log"
Jul 02 19:45:06.802559 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:06.802525    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/1.log"
Jul 02 19:45:06.887251 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:06.887165    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ddfm5_78211ab9-79b8-43d1-902a-fd8b7cb524e0/payload-processing/0.log"
Jul 02 19:45:07.566678 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:07.566643    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8lk8m_8cc512c5-ec30-4764-b576-8c8b7f183298/serve-healthcheck-canary/0.log"
Jul 02 19:45:08.211987 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:08.211949    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v6p5t_9bae5299-39e9-4210-a777-a1ed57e4b596/kube-rbac-proxy/0.log"
Jul 02 19:45:08.235815 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:08.235790    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v6p5t_9bae5299-39e9-4210-a777-a1ed57e4b596/exporter/0.log"
Jul 02 19:45:08.260843 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:08.260817    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v6p5t_9bae5299-39e9-4210-a777-a1ed57e4b596/extractor/0.log"
Jul 02 19:45:09.675472 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:09.675446    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-88s92/perf-node-gather-daemonset-5q6md"
Jul 02 19:45:10.863268 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:10.863242    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6464bbbddb-g2vth_4ce4beab-3cfb-4db3-847a-ad22302b6469/manager/0.log"
Jul 02 19:45:10.911694 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:10.911669    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r4hsl_9fda2112-24d7-42c5-a887-74e060e85ae4/server/0.log"
Jul 02 19:45:10.981792 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:10.981762    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8hgmc_cfe96aa1-7ea1-4f2a-8d33-f3c3f6ed9323/manager/0.log"
Jul 02 19:45:11.020589 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:11.020564    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb89c69-4rjpk_8954e82e-6c64-4c92-814d-702dc9889237/manager/0.log"
Jul 02 19:45:11.056591 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:11.056563    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb89c69-4rjpk_8954e82e-6c64-4c92-814d-702dc9889237/copy-manifests/0.log"
Jul 02 19:45:11.819709 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:11.819622    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb67864b99_cedd2fc4-ac14-437d-94dd-61a3fc625a1a/istio-proxy/0.log"
Jul 02 19:45:11.936143 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:11.936108    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ks5b4_a87ccab5-02ad-45f9-93b2-6681039a7b99/discovery/0.log"
Jul 02 19:45:11.969238 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:11.969207    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-578747f885-tgqz8_88504682-7871-4c59-95cb-cb412de7f35c/kube-auth-proxy/0.log"
Jul 02 19:45:11.992456 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:11.992420    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/0.log"
Jul 02 19:45:12.019790 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:12.019763    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/1.log"
Jul 02 19:45:12.087941 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:12.087916    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ddfm5_78211ab9-79b8-43d1-902a-fd8b7cb524e0/payload-processing/0.log"
Jul 02 19:45:13.174438 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:13.174320    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb67864b99_cedd2fc4-ac14-437d-94dd-61a3fc625a1a/istio-proxy/0.log"
Jul 02 19:45:13.279699 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:13.279667    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ks5b4_a87ccab5-02ad-45f9-93b2-6681039a7b99/discovery/0.log"
Jul 02 19:45:13.314645 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:13.314622    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-578747f885-tgqz8_88504682-7871-4c59-95cb-cb412de7f35c/kube-auth-proxy/0.log"
Jul 02 19:45:13.338217 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:13.338190    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/0.log"
Jul 02 19:45:13.364586 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:13.364551    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-5h8rj_da9439cf-a90e-4a70-9b66-5ccb6f5feb9e/istio-proxy/1.log"
Jul 02 19:45:13.431965 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:13.431876    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ddfm5_78211ab9-79b8-43d1-902a-fd8b7cb524e0/payload-processing/0.log"
Jul 02 19:45:17.408719 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:17.408689    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-4n2qw_f6e90a60-b687-4089-88de-77a80339e0f1/migrator/0.log"
Jul 02 19:45:17.432738 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:17.432710    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-4n2qw_f6e90a60-b687-4089-88de-77a80339e0f1/graceful-termination/0.log"
Jul 02 19:45:19.274387 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:19.274357    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9qfc_a3820882-8598-456f-9604-0306e0956be5/kube-multus-additional-cni-plugins/0.log"
Jul 02 19:45:19.300710 ip-10-0-135-120 kubenswrapper[2584]: I0702 19:45:19.300682    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9qfc_a3820882-8598-456f-9604-0306e0956be5/egress-router-binary-copy/0.log"