Jul 08 19:03:10.403009 ip-10-0-130-212 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 08 19:03:10.403024 ip-10-0-130-212 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 08 19:03:10.403034 ip-10-0-130-212 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 08 19:03:10.403637 ip-10-0-130-212 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 08 19:03:20.652088 ip-10-0-130-212 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 08 19:03:20.652113 ip-10-0-130-212 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c13268b2d9734755bcd8e43244a36ef1 --
Jul 08 19:05:26.214412 ip-10-0-130-212 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 19:05:26.635620 ip-10-0-130-212 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 19:05:26.635620 ip-10-0-130-212 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 19:05:26.635620 ip-10-0-130-212 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 19:05:26.635620 ip-10-0-130-212 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 19:05:26.635620 ip-10-0-130-212 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 19:05:26.637421 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.637328    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643696    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643722    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643727    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643731    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643734    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643737    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 19:05:26.643728 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643741    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643744    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643747    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643749    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643752    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643754    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643757    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643759    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643762    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643764    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643767    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643771    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643775    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643778    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643780    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643783    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643785    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643788    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643790    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643793    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 19:05:26.644001 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643795    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643798    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643800    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643803    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643805    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643815    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643818    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643820    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643823    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643825    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643828    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643832    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643835    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643838    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643842    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643846    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643849    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643852    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643855    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643858    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 19:05:26.644499 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643861    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643865    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643867    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643870    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643873    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643876    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643878    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643881    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643884    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643886    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643889    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643891    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643894    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643897    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643899    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643902    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643904    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643907    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643909    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 19:05:26.644989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643912    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643914    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643917    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643920    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643922    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643925    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643928    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643931    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643933    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643936    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643939    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643941    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643944    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643946    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643949    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643951    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643953    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643956    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643959    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 19:05:26.645463 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643962    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643964    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643967    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643969    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643972    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643975    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643980    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643984    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643987    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643989    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643992    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643994    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643997    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.643999    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644002    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644754    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644762    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644765    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644768    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644773    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 19:05:26.645926 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644777    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644781    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644785    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644788    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644791    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644793    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644796    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644799    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644801    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644804    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644807    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644809    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644813    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644815    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644818    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644821    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644823    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644826    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644828    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644831    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 19:05:26.646428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644833    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644836    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644839    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644841    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644844    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644847    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644850    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644852    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644855    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644857    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644860    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644863    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644866    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644870    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644873    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644875    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644878    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644881    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644883    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644886    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 19:05:26.646941 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644889    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644892    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644894    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644897    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644899    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644902    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644904    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644908    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644911    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644914    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644917    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644919    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644921    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644924    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644926    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644929    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644931    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644934    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644936    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 19:05:26.647492 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644939    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644941    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644944    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644948    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644951    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644955    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644957    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644961    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644964    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644966    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644969    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644971    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644974    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644976    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644979    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644981    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644984    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644986    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644989    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644992    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 19:05:26.647963 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644994    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.644997    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645000    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645002    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645005    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645007    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645010    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645013    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645015    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645018    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645020    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645023    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645025    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645027    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645030    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645478    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645484    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645487    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645490    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645493    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 19:05:26.648477 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645495    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645498    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645501    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645503    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645506    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645509    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645511    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645514    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645516    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645519    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645521    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645524    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645527    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645529    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645532    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645534    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645537    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645540    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645542    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645548    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 19:05:26.648995 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645550    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645553    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645555    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645558    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645561    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645563    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645566    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645568    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645571    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645574    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645576    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645579    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645582    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645584    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645586    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645589    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645592    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645594    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645597    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645599    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645602    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 19:05:26.649489 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645604    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645607    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645609    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645612    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645615    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645618    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645620    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645622    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645625    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645627    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645633    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645637    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645640    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645643    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645646    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645648    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645651    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645653    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645656    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 19:05:26.649998 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645658    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645661    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645663    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645666    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645668    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645673    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645675    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645678    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645682    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645684    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645687    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645689    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645692    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645694    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645697    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645699    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645702    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645705    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645707    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645710    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 19:05:26.650481 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645712    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645715    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645718    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645723    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645725    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645728    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645730    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645733    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645735    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645737    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645740    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645744    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645746    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.645749    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645824    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645833    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645839    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645844    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645848    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645851    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 19:05:26.650959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645855    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645860    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645863    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645866    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645869    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645872    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645875    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645878    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645881    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645884    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645887    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645890    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645895    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645897    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645900    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645905    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645908    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645911    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645914    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645917    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645921    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645924    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645927    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645931    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 19:05:26.651474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645935    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645938    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645942    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645945    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645948    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645951    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645954    2563 flags.go:64] FLAG: --enable-server="true"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645957    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645962    2563 flags.go:64] FLAG: --event-burst="100"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645965    2563 flags.go:64] FLAG: --event-qps="50"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645968    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645971    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645974    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645978    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645981    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645984    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645986    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645989    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645992    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645996    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.645999    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646002    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646005    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646009    2563 flags.go:64] FLAG: --feature-gates=""
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646013    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 19:05:26.652047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646016    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646019    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646022    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646024    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646027    2563 flags.go:64] FLAG: --help="false"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646030    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-130-212.ec2.internal"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646034    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646037    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646040    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646043    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646046    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646048    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646051    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646054    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646056    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646059    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646062    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646065    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646067    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646070    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646073    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646076    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646092    2563 flags.go:64] FLAG: --lock-file=""
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646095    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 19:05:26.652710 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646098    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646101    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646106    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646110    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646113    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646115    2563 flags.go:64] FLAG: --logging-format="text"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646119    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646123    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646125    2563 flags.go:64] FLAG: --manifest-url=""
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646128    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646132    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646135    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646139    2563 flags.go:64] FLAG: --max-pods="110"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646142    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646147    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646149    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646152    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646155    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646158    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646161    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646168    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646171    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646174    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 19:05:26.653320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646177    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646180    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646187    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646189    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646192    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646195    2563 flags.go:64] FLAG: --port="10250"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646198    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646201    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-029268771d95179c7"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646204    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646207    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646210    2563 flags.go:64] FLAG: --register-node="true"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646213    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646216    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646219    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646222    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646227    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646231    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646234    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646236    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646239    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646242    2563 flags.go:64] FLAG: --runonce="false"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646245    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646248    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646251    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646256    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 19:05:26.653895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646258    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646261    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646264    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646267    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646270    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646273    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646276    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646279    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646281    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646284    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646287    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646292    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646295    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646298    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646303    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646305    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646308    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646311    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646313    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646316    2563 flags.go:64] FLAG: --v="2"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646321    2563 flags.go:64] FLAG: --version="false"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646326    2563 flags.go:64] FLAG: --vmodule=""
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646331    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646334    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646455    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 19:05:26.654541 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646459    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646461    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646464    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646467    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646469    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646472    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646478    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646480    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646483    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646486    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646488    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646491    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646493    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646496    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646498    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646501    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646503    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646506    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646508    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 19:05:26.655163 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646513    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646516    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646519    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646522    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646525    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646527    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646530    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646532    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646535    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646537    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646549    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646551    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646555    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646558    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646561    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646563    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646566    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646569    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646571    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 19:05:26.655626 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646575    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646578    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646580    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646583    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646585    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646588    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646590    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646593    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646595    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646598    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646600    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646603    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646605    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646608    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646610    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646613    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646615    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646617    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646620    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646622    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 19:05:26.656123 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646625    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646627    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646630    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646633    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646636    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646639    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646641    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646644    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646646    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646649    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646652    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646654    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646658    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646660    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646663    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646666    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646668    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646671    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646673    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646677    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 19:05:26.656629 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646680    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646683    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646686    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646688    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646690    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646693    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646695    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646698    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646700    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646703    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646705    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646708    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646710    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646712    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646715    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646719    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646721    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646725    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646728    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 19:05:26.657134 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.646730    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.646736    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.653960    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.653979    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.654708    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.654716    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654777    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654782    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654786    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654789    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654792    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654795    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654798    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654801    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654804    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 19:05:26.657594 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654807    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654809    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654812    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654815    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654818    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654821    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654823    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654827    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654831    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654834    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654837    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654839    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654842    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654845    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654848    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654850    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654853    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654856    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654858    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 19:05:26.658022 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654860    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654864    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654867    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654870    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654874    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654878    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654880    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654883    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654885    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654888    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654891    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654893    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654896    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654898    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654901    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654904    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654906    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654909    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654911    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654914    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 19:05:26.658523 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654917    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654919    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654921    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654924    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654927    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654930    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654932    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654935    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654937    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654940    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654943    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654945    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654948    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654951    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654954    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654956    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654959    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654962    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654964    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654966    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 19:05:26.659049 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654969    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654971    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654974    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654976    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654980    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654983    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654986    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654989    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654991    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654994    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654997    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.654999    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655002    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655004    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655007    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655009    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655012    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655015    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655017    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 19:05:26.659572 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655020    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655023    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655025    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655028    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655030    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655033    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655036    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655039    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655042    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655044    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655047    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655049    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.655054    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655191    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655197    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 19:05:26.660043 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655200    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655203    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655206    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655208    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655211    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655214    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655217    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655219    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655222    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655226    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655230    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655232    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655235    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655238    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655241    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655262    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655265    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655268    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655271    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 19:05:26.660456 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655274    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655276    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655279    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655282    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655285    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655288    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655291    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655294    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655296    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655299    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655301    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655304    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655306    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655309    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655311    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655314    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655316    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655318    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655321    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655323    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 19:05:26.660910 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655326    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655328    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655330    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655333    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655335    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655338    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655341    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655343    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655346    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655348    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655350    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655353    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655356    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655358    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655361    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655363    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655366    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655368    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655371    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655375    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 19:05:26.661412 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655387    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655391    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655394    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655397    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655400    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655402    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655405    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655408    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655410    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655413    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655416    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655418    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655420    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655423    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655426    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655428    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655430    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655433    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655436    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 19:05:26.661889 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655439    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655441    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655444    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655446    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655449    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655451    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655454    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655456    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655459    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655461    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655464    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655467    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655471    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655473    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655478    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655480    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655483    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655486    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 19:05:26.662365 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:26.655489    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.655493    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.655631    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.657948    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.659445    2563 server.go:1019] "Starting client certificate rotation"
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.659551    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.660321    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 19:05:26.662833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.662724    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 19:05:26.667786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.667758    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 19:05:26.679102 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.679067    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 08 19:05:26.681873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.681849    2563 log.go:25] "Validated CRI v1 image API"
Jul 08 19:05:26.682661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.682643    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 19:05:26.685202 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.685174    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ca7fb5bc-5fa7-4bca-9c6e-6eaed95b96c5:/dev/nvme0n1p3 e7a41d81-d4a6-4cc1-98ea-8031904e522d:/dev/nvme0n1p4]
Jul 08 19:05:26.685275 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.685200    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 19:05:26.687682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.687655    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 19:05:26.690669 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.690560    2563 manager.go:216] Machine: {Timestamp:2026-07-08 19:05:26.689170677 +0000 UTC m=+0.353733218 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096646 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e728ffeda35c3831c8453eb652276 SystemUUID:ec2e728f-feda-35c3-831c-8453eb652276 BootID:c13268b2-d973-4755-bcd8-e43244a36ef1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:97:fc:33:97:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:97:fc:33:97:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:4d:a3:90:ac:7b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 19:05:26.690669 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.690664    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 19:05:26.690775 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.690747    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 19:05:26.694108 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.694067    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 19:05:26.694269 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.694112    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-212.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 19:05:26.694884 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.694873    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 19:05:26.694916 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.694886    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 19:05:26.694916 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.694894    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 19:05:26.695619 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.695607    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 19:05:26.697176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.697165    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 19:05:26.700713 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.700700    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 19:05:26.701031 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701021    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 19:05:26.701060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701054    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 19:05:26.701106 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701071    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 19:05:26.701106 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701105    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 19:05:26.701170 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701161    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 08 19:05:26.701208 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701178    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 19:05:26.701601 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701589    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 19:05:26.701639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.701609    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 19:05:26.703998 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.703981    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 19:05:26.706132 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.706119    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 19:05:26.706183 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.706152    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 19:05:26.707475 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707463    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707480    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707487    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707493    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707504    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707509    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707517    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 19:05:26.707525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707522    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 19:05:26.707698 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707530    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 19:05:26.707698 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707536    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 19:05:26.707698 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707544    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 19:05:26.707698 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.707549    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 19:05:26.708288 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.708277    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 19:05:26.708328 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.708289    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 19:05:26.712829 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.712754    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 19:05:26.712829 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.712803    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 19:05:26.712948 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.712911    2563 server.go:1268] "Started kubelet"
Jul 08 19:05:26.712981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.712944    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-212.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 19:05:26.713010 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.712989    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 19:05:26.713416 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.713356    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 19:05:26.713495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.713466    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 19:05:26.714316 ip-10-0-130-212 systemd[1]: Started Kubernetes Kubelet.
Jul 08 19:05:26.714940 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.714847    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 19:05:26.715475 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.715451    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 19:05:26.719594 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.719574    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 19:05:26.721449 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.720463    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-212.ec2.internal.18c065ee534ce070  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-212.ec2.internal,UID:ip-10-0-130-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-212.ec2.internal,},FirstTimestamp:2026-07-08 19:05:26.71288536 +0000 UTC m=+0.377447901,LastTimestamp:2026-07-08 19:05:26.71288536 +0000 UTC m=+0.377447901,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-212.ec2.internal,}"
Jul 08 19:05:26.722022 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722006    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 19:05:26.722125 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722057    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 19:05:26.722125 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.722107    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:26.722774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722758    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 19:05:26.722774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722775    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 19:05:26.722903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722758    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 19:05:26.722903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722877    2563 factory.go:55] Registering systemd factory
Jul 08 19:05:26.722903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722886    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 19:05:26.722903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722901    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 19:05:26.723067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.722894    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 08 19:05:26.723153 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.723141    2563 factory.go:153] Registering CRI-O factory
Jul 08 19:05:26.723153 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.723153    2563 factory.go:223] Registration of the crio container factory successfully
Jul 08 19:05:26.723256 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.723196    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 19:05:26.723256 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.723226    2563 factory.go:103] Registering Raw factory
Jul 08 19:05:26.723256 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.723236    2563 manager.go:1196] Started watching for new ooms in manager
Jul 08 19:05:26.723554 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.723543    2563 manager.go:349] Starting recovery of all containers
Jul 08 19:05:26.730609 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.730585    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-km5fg"
Jul 08 19:05:26.734457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.734328    2563 manager.go:354] Recovery completed
Jul 08 19:05:26.734799 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.734772    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 19:05:26.734872 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.734796    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 19:05:26.739302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.739283    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:26.741616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.741600    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:26.741684 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.741628    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:26.741684 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.741640    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:26.742281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.742152    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 19:05:26.742281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.742161    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:26.742281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.742165    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 19:05:26.742281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.742184    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 19:05:26.743096 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743066    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:26.743225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743130    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:26.743225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743148    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:26.743375 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743359    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-km5fg"
Jul 08 19:05:26.743565 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743552    2563 policy_none.go:49] "None policy: Start"
Jul 08 19:05:26.743612 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743570    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 19:05:26.743612 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.743580    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 19:05:26.744669 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.744584    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-212.ec2.internal.18c065ee550343f1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-212.ec2.internal,UID:ip-10-0-130-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-212.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-212.ec2.internal,},FirstTimestamp:2026-07-08 19:05:26.741615601 +0000 UTC m=+0.406178146,LastTimestamp:2026-07-08 19:05:26.741615601 +0000 UTC m=+0.406178146,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-212.ec2.internal,}"
Jul 08 19:05:26.745277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.745258    2563 policy_none.go:47] "Start"
Jul 08 19:05:26.777714 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.777698    2563 manager.go:337] "Starting Device Plugin manager"
Jul 08 19:05:26.777825 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.777734    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 19:05:26.777825 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.777743    2563 server.go:87] "Starting device plugin registration server"
Jul 08 19:05:26.778058 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.778046    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 19:05:26.778117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.778062    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 19:05:26.778175 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.778158    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 19:05:26.778258 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.778242    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 19:05:26.778258 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.778256    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 19:05:26.778836 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.778810    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 19:05:26.778904 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.778871    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:26.878828 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.878789    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:26.880220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.880026    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:26.880220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.880058    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:26.880220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.880070    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:26.880220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.880120    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-212.ec2.internal"
Jul 08 19:05:26.889721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.889644    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-212.ec2.internal"
Jul 08 19:05:26.889721 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.889680    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-212.ec2.internal\": node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:26.908704 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.908669    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:26.922146 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.922116    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 19:05:26.923562 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.923545    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 19:05:26.923676 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.923585    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 19:05:26.923676 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.923611    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 19:05:26.923676 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:26.923647    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 19:05:26.925644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:26.925627    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 19:05:27.009063 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.009021    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.024369 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.024326    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"]
Jul 08 19:05:27.024442 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.024415    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:27.025409 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.025393    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:27.025462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.025425    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:27.025462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.025434    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:27.026777 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.026765    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:27.026889 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.026876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.026931 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.026903    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:27.027516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.027503    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:27.027516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.027509    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:27.027615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.027527    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:27.027615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.027534    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:27.027615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.027549    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:27.027615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.027537    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:27.028727 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.028712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.028778 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.028741    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 19:05:27.029404 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.029390    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 19:05:27.029459 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.029414    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 19:05:27.029459 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.029424    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeHasSufficientPID"
Jul 08 19:05:27.058945 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.058912    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-212.ec2.internal\" not found" node="ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.063498 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.063481    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-212.ec2.internal\" not found" node="ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.110030 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.109997    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.124568 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.124547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0213be8aee70fe73b207187d672b3779-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal\" (UID: \"0213be8aee70fe73b207187d672b3779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.124642 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.124575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0213be8aee70fe73b207187d672b3779-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal\" (UID: \"0213be8aee70fe73b207187d672b3779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.124642 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.124589    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/991d6b5a2a481334cc1eaf58c5d1d435-config\") pod \"kube-apiserver-proxy-ip-10-0-130-212.ec2.internal\" (UID: \"991d6b5a2a481334cc1eaf58c5d1d435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.210609 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.210537    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.224866 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.224841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0213be8aee70fe73b207187d672b3779-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal\" (UID: \"0213be8aee70fe73b207187d672b3779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.224949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.224870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0213be8aee70fe73b207187d672b3779-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal\" (UID: \"0213be8aee70fe73b207187d672b3779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.224949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.224886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/991d6b5a2a481334cc1eaf58c5d1d435-config\") pod \"kube-apiserver-proxy-ip-10-0-130-212.ec2.internal\" (UID: \"991d6b5a2a481334cc1eaf58c5d1d435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.224949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.224924    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/991d6b5a2a481334cc1eaf58c5d1d435-config\") pod \"kube-apiserver-proxy-ip-10-0-130-212.ec2.internal\" (UID: \"991d6b5a2a481334cc1eaf58c5d1d435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.224949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.224933    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0213be8aee70fe73b207187d672b3779-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal\" (UID: \"0213be8aee70fe73b207187d672b3779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.224949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.224943    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0213be8aee70fe73b207187d672b3779-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal\" (UID: \"0213be8aee70fe73b207187d672b3779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.310967 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.310926    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.362454 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.362424    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.367388 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.367369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:27.411976 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.411945    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.512381 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.512301    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.612827 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.612789    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.659208 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.659182    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 19:05:27.659716 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.659331    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 19:05:27.713670 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.713630    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.746157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.746110    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 19:00:26 +0000 UTC" deadline="2028-04-15 20:24:47.275166182 +0000 UTC"
Jul 08 19:05:27.746157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.746145    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15529h19m19.529024592s"
Jul 08 19:05:27.814484 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.814395    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.914880 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:27.914849    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-212.ec2.internal\" not found"
Jul 08 19:05:27.954865 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:27.954835    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0213be8aee70fe73b207187d672b3779.slice/crio-d02e1f2bfb6ae56009108f9f878088bd61651f60e0f02e444b1339d8013ea98b WatchSource:0}: Error finding container d02e1f2bfb6ae56009108f9f878088bd61651f60e0f02e444b1339d8013ea98b: Status 404 returned error can't find the container with id d02e1f2bfb6ae56009108f9f878088bd61651f60e0f02e444b1339d8013ea98b
Jul 08 19:05:27.955170 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:27.955148    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991d6b5a2a481334cc1eaf58c5d1d435.slice/crio-d1b242770795eb8e73440a184c2b1ec4017f8d800f6f4eef365664e0895979ea WatchSource:0}: Error finding container d1b242770795eb8e73440a184c2b1ec4017f8d800f6f4eef365664e0895979ea: Status 404 returned error can't find the container with id d1b242770795eb8e73440a184c2b1ec4017f8d800f6f4eef365664e0895979ea
Jul 08 19:05:27.959197 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.959175    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 19:05:27.963154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.963136    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 19:05:27.988187 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:27.988155    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 19:05:28.022539 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.022512    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:28.034147 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.034127    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 19:05:28.035032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.035019    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal"
Jul 08 19:05:28.043182 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.043167    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 19:05:28.177147 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.177053    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 19:05:28.701689 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.701661    2563 apiserver.go:52] "Watching apiserver"
Jul 08 19:05:28.709619 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.709591    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 19:05:28.709944 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.709924    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-wvxjv","kube-system/konnectivity-agent-f2mkg","kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jxqwh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal","openshift-multus/multus-additional-cni-plugins-gmmcp","openshift-network-diagnostics/network-check-target-vrp9v","openshift-network-operator/iptables-alerter-ntjss","kube-system/global-pull-secret-syncer-4sk5h","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw","openshift-dns/node-resolver-g2rm5","openshift-image-registry/node-ca-t4wlb","openshift-multus/multus-wtcj7","openshift-multus/network-metrics-daemon-kxb2x"]
Jul 08 19:05:28.711571 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.711550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:28.711677 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.711625    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:28.712871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.712851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.714178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.714159    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:28.714270 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.714227    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:28.715180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.715159    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.715180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.715176    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.715315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.715216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k5nx8\""
Jul 08 19:05:28.715513 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.715492    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.717587 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.717572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.718833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.718814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.720033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.720016    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.720127 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.720076    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:28.721767 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.721747    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.723135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.723120    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 19:05:28.723531 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.723515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.724592 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.724572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.726342 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.726327    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.728363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.728342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.729770 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.729142    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysctl-conf\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djwqn\" (UniqueName: \"kubernetes.io/projected/71b5f67b-07b0-4a25-80aa-d196ac472005-kube-api-access-djwqn\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e55a4251-70cc-42b2-bf2b-372748f73320-host-slash\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-system-cni-dir\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvj77\" (UniqueName: \"kubernetes.io/projected/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-kube-api-access-qvj77\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41698256-87d5-433e-9e4f-a3f7de1fbfc3-cni-binary-copy\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.730709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-kubelet\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b2f1675-9c4c-40dc-893e-6dd1ed09aecb-agent-certs\") pod \"konnectivity-agent-f2mkg\" (UID: \"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb\") " pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysctl-d\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-systemd\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwdpv\" (UniqueName: \"kubernetes.io/projected/e55a4251-70cc-42b2-bf2b-372748f73320-kube-api-access-mwdpv\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-os-release\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-multus-certs\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b2f1675-9c4c-40dc-893e-6dd1ed09aecb-konnectivity-ca\") pod \"konnectivity-agent-f2mkg\" (UID: \"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb\") " pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x7b9\" (UniqueName: \"kubernetes.io/projected/11b1cfea-7567-45ce-a015-d1947a311e07-kube-api-access-4x7b9\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrz8n\" (UniqueName: \"kubernetes.io/projected/69ca6196-d827-45d8-985d-b0d8ffcd0f57-kube-api-access-qrz8n\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-sys\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-cnibin\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysconfig\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b5f67b-07b0-4a25-80aa-d196ac472005-tmp\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e55a4251-70cc-42b2-bf2b-372748f73320-iptables-alerter-script\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.731072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.730985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-conf-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b1cfea-7567-45ce-a015-d1947a311e07-tmp-dir\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-run\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-lib-modules\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-var-lib-kubelet\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-daemon-config\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r47m\" (UniqueName: \"kubernetes.io/projected/41698256-87d5-433e-9e4f-a3f7de1fbfc3-kube-api-access-8r47m\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11b1cfea-7567-45ce-a015-d1947a311e07-hosts-file\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-kubernetes\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-host\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-tuned\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cnibin\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.731842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-system-cni-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-modprobe-d\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-kubelet-config\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-dbus\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-cni-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-k8s-cni-cncf-io\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-cni-bin\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-cni-multus\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-hostroot\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-etc-kubernetes\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-os-release\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-socket-dir-parent\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.732463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.731686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-netns\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.734911 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.734887    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sw7t4\""
Jul 08 19:05:28.737595 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.737573    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tt58w\""
Jul 08 19:05:28.737738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.737598    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 19:05:28.737738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.737629    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 19:05:28.737906 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.737839    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 19:05:28.737962 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.737915    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.738008 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.737987    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.738111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.738077    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.738221 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.738206    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 19:05:28.738281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.738229    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 19:05:28.738484 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.738468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cphpk\""
Jul 08 19:05:28.739217 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.739202    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.741368 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.741349    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rb4bk\""
Jul 08 19:05:28.744821 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.744804    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v6c9m\""
Jul 08 19:05:28.745261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.745243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z47mx\""
Jul 08 19:05:28.745338 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.745267    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.760370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.760343    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 19:05:28.760594 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.760579    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.760774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.760760    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 19:05:28.761288 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.761268    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 19:05:28.761372 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.761305    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 19:05:28.773717 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.773693    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.774049 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.774035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.774372 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.774354    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775235    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775255    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775405    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z7dcr\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775436    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 19:05:28.775747 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.775644    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 19:05:28.785126 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.785104    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6g28z\""
Jul 08 19:05:28.798845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.798814    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 19:05:28.800907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.800884    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b9wlp"
Jul 08 19:05:28.807948 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.807920    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b9wlp"
Jul 08 19:05:28.824252 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.824227    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 19:05:28.831862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.831829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e55a4251-70cc-42b2-bf2b-372748f73320-host-slash\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.831862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.831872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-system-cni-dir\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.831898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41698256-87d5-433e-9e4f-a3f7de1fbfc3-cni-binary-copy\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.831920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b2f1675-9c4c-40dc-893e-6dd1ed09aecb-agent-certs\") pod \"konnectivity-agent-f2mkg\" (UID: \"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb\") " pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.831946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-etc-selinux\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.831969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-ovn\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-run-ovn-kubernetes\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.832094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-os-release\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-multus-certs\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x7b9\" (UniqueName: \"kubernetes.io/projected/11b1cfea-7567-45ce-a015-d1947a311e07-kube-api-access-4x7b9\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrz8n\" (UniqueName: \"kubernetes.io/projected/69ca6196-d827-45d8-985d-b0d8ffcd0f57-kube-api-access-qrz8n\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-env-overrides\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b5f67b-07b0-4a25-80aa-d196ac472005-tmp\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlknv\" (UniqueName: \"kubernetes.io/projected/4fb99271-9db5-4d6a-b3a9-152683b48ca5-kube-api-access-tlknv\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b1cfea-7567-45ce-a015-d1947a311e07-tmp-dir\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-lib-modules\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-log-socket\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832288    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e55a4251-70cc-42b2-bf2b-372748f73320-host-slash\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-multus-certs\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832385    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-os-release\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.832450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832438    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-system-cni-dir\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.833005 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832861    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 19:05:28.833005 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832914    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41698256-87d5-433e-9e4f-a3f7de1fbfc3-cni-binary-copy\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.832299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-daemon-config\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r47m\" (UniqueName: \"kubernetes.io/projected/41698256-87d5-433e-9e4f-a3f7de1fbfc3-kube-api-access-8r47m\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-tuned\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833217    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-lib-modules\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fb99271-9db5-4d6a-b3a9-152683b48ca5-host\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-registration-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833313    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b1cfea-7567-45ce-a015-d1947a311e07-tmp-dir\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-ovnkube-config\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cnibin\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysctl-d\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-systemd\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833615    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cnibin\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-systemd-units\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-slash\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.833982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9c3b6fe-289b-427e-bb03-831dae993b10-ovn-node-metrics-cert\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833684    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-systemd\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-kubelet-config\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-dbus\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833741    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysctl-d\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-etc-kubernetes\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833772    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-kubelet-config\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833813    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-etc-kubernetes\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833630    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-daemon-config\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833821    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-sys-fs\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.834841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.833845    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-dbus\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.838399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b5f67b-07b0-4a25-80aa-d196ac472005-tmp\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.838816    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.838970    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-tuned\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.838993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-node-log\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-os-release\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysctl-conf\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djwqn\" (UniqueName: \"kubernetes.io/projected/71b5f67b-07b0-4a25-80aa-d196ac472005-kube-api-access-djwqn\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-device-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92g98\" (UniqueName: \"kubernetes.io/projected/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-kube-api-access-92g98\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-kubelet\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-var-lib-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvj77\" (UniqueName: \"kubernetes.io/projected/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-kube-api-access-qvj77\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-kubelet\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-sys\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-socket-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.840100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-run-netns\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-ovnkube-script-lib\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwdpv\" (UniqueName: \"kubernetes.io/projected/e55a4251-70cc-42b2-bf2b-372748f73320-kube-api-access-mwdpv\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b2f1675-9c4c-40dc-893e-6dd1ed09aecb-konnectivity-ca\") pod \"konnectivity-agent-f2mkg\" (UID: \"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb\") " pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-etc-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-cni-bin\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-969vc\" (UniqueName: \"kubernetes.io/projected/a9c3b6fe-289b-427e-bb03-831dae993b10-kube-api-access-969vc\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-cnibin\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysconfig\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fb99271-9db5-4d6a-b3a9-152683b48ca5-serviceca\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e55a4251-70cc-42b2-bf2b-372748f73320-iptables-alerter-script\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-conf-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-cni-netd\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.840616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11b1cfea-7567-45ce-a015-d1947a311e07-hosts-file\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-kubernetes\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-host\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-systemd\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-run\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-var-lib-kubelet\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.839995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-system-cni-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-modprobe-d\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-cni-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-k8s-cni-cncf-io\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-cni-bin\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-cni-multus\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-hostroot\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-socket-dir-parent\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-netns\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.840296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.841507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.841206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.841507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.841286    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-os-release\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.841507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.841397    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysctl-conf\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.841908 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.841888    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-kubelet\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.842009 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.841957    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-sys\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842337    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842401    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-host\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842390    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-kubernetes\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.842599    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.842619    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.842638    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842768    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-var-lib-kubelet\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.842777    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842814    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-run\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.842857    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:29.342836584 +0000 UTC m=+3.007399130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842865    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11b1cfea-7567-45ce-a015-d1947a311e07-hosts-file\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842886    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b2f1675-9c4c-40dc-893e-6dd1ed09aecb-konnectivity-ca\") pod \"konnectivity-agent-f2mkg\" (UID: \"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb\") " pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842953    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-system-cni-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.842967    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-modprobe-d\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843025    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-k8s-cni-cncf-io\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843036    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-conf-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843108    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-run-netns\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.843136    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.843189    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:05:29.34317306 +0000 UTC m=+3.007735599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843243    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/71b5f67b-07b0-4a25-80aa-d196ac472005-etc-sysconfig\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843302    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-cni-dir\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-cni-bin\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843381    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-host-var-lib-cni-multus\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843414    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-hostroot\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843457    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-cnibin\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843499    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41698256-87d5-433e-9e4f-a3f7de1fbfc3-multus-socket-dir-parent\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:28.843547    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:29.343532992 +0000 UTC m=+3.008095533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:28.843786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843674    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e55a4251-70cc-42b2-bf2b-372748f73320-iptables-alerter-script\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.844351 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.843889    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.847976 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.847648    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r47m\" (UniqueName: \"kubernetes.io/projected/41698256-87d5-433e-9e4f-a3f7de1fbfc3-kube-api-access-8r47m\") pod \"multus-wtcj7\" (UID: \"41698256-87d5-433e-9e4f-a3f7de1fbfc3\") " pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:28.847976 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.847933    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b2f1675-9c4c-40dc-893e-6dd1ed09aecb-agent-certs\") pod \"konnectivity-agent-f2mkg\" (UID: \"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb\") " pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:28.848281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.848261    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrz8n\" (UniqueName: \"kubernetes.io/projected/69ca6196-d827-45d8-985d-b0d8ffcd0f57-kube-api-access-qrz8n\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:28.849134 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.849057    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x7b9\" (UniqueName: \"kubernetes.io/projected/11b1cfea-7567-45ce-a015-d1947a311e07-kube-api-access-4x7b9\") pod \"node-resolver-g2rm5\" (UID: \"11b1cfea-7567-45ce-a015-d1947a311e07\") " pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:28.852897 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.852874    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djwqn\" (UniqueName: \"kubernetes.io/projected/71b5f67b-07b0-4a25-80aa-d196ac472005-kube-api-access-djwqn\") pod \"tuned-jxqwh\" (UID: \"71b5f67b-07b0-4a25-80aa-d196ac472005\") " pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:28.853213 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.853151    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvj77\" (UniqueName: \"kubernetes.io/projected/1677a6e8-b5a9-42a2-8a07-ce4b33ed3328-kube-api-access-qvj77\") pod \"multus-additional-cni-plugins-gmmcp\" (UID: \"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328\") " pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:28.854135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.854047    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwdpv\" (UniqueName: \"kubernetes.io/projected/e55a4251-70cc-42b2-bf2b-372748f73320-kube-api-access-mwdpv\") pod \"iptables-alerter-ntjss\" (UID: \"e55a4251-70cc-42b2-bf2b-372748f73320\") " pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:28.856677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.856658    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 19:05:28.928268 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.928222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal" event={"ID":"991d6b5a2a481334cc1eaf58c5d1d435","Type":"ContainerStarted","Data":"d1b242770795eb8e73440a184c2b1ec4017f8d800f6f4eef365664e0895979ea"}
Jul 08 19:05:28.929350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.929321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal" event={"ID":"0213be8aee70fe73b207187d672b3779","Type":"ContainerStarted","Data":"d02e1f2bfb6ae56009108f9f878088bd61651f60e0f02e444b1339d8013ea98b"}
Jul 08 19:05:28.941029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-systemd-units\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-slash\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9c3b6fe-289b-427e-bb03-831dae993b10-ovn-node-metrics-cert\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-systemd-units\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941121    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-slash\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-sys-fs\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-node-log\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941194    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-sys-fs\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-device-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941218    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-node-log\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92g98\" (UniqueName: \"kubernetes.io/projected/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-kube-api-access-92g98\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941245    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-kubelet\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941252    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-device-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-var-lib-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941286    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-kubelet\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-socket-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-run-netns\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-ovnkube-script-lib\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-etc-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-cni-bin\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-969vc\" (UniqueName: \"kubernetes.io/projected/a9c3b6fe-289b-427e-bb03-831dae993b10-kube-api-access-969vc\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fb99271-9db5-4d6a-b3a9-152683b48ca5-serviceca\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941409    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-run-netns\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-cni-netd\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-systemd\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941458    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-var-lib-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.941611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-etc-selinux\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941557    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-socket-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-ovn\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941595    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-ovn\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-run-ovn-kubernetes\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941623    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-run-ovn-kubernetes\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941670    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941706    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-etc-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-cni-bin\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-env-overrides\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlknv\" (UniqueName: \"kubernetes.io/projected/4fb99271-9db5-4d6a-b3a9-152683b48ca5-kube-api-access-tlknv\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-log-socket\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fb99271-9db5-4d6a-b3a9-152683b48ca5-host\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-registration-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.941879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-ovnkube-config\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942034    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fb99271-9db5-4d6a-b3a9-152683b48ca5-serviceca\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942108    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-cni-netd\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942145    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-systemd\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942168    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942239    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-etc-selinux\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942238    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-ovnkube-script-lib\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942270    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-run-openvswitch\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942291    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a9c3b6fe-289b-427e-bb03-831dae993b10-log-socket\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942403    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-ovnkube-config\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942456    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fb99271-9db5-4d6a-b3a9-152683b48ca5-host\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942502    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-registration-dir\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:28.942961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.942501    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a9c3b6fe-289b-427e-bb03-831dae993b10-env-overrides\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.945495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.945467    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9c3b6fe-289b-427e-bb03-831dae993b10-ovn-node-metrics-cert\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.954419 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.954360    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlknv\" (UniqueName: \"kubernetes.io/projected/4fb99271-9db5-4d6a-b3a9-152683b48ca5-kube-api-access-tlknv\") pod \"node-ca-t4wlb\" (UID: \"4fb99271-9db5-4d6a-b3a9-152683b48ca5\") " pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:28.954532 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.954429    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-969vc\" (UniqueName: \"kubernetes.io/projected/a9c3b6fe-289b-427e-bb03-831dae993b10-kube-api-access-969vc\") pod \"ovnkube-node-wvxjv\" (UID: \"a9c3b6fe-289b-427e-bb03-831dae993b10\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:28.954856 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:28.954836    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92g98\" (UniqueName: \"kubernetes.io/projected/e2d05400-7a62-4e6e-bb7b-97d3c4eaff92-kube-api-access-92g98\") pod \"aws-ebs-csi-driver-node-q7btw\" (UID: \"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:29.023194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.023163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g2rm5"
Jul 08 19:05:29.032574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.032375    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jxqwh"
Jul 08 19:05:29.041400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.041371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ntjss"
Jul 08 19:05:29.047582 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.047556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmmcp"
Jul 08 19:05:29.054719 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.054695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wtcj7"
Jul 08 19:05:29.062026 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.062003    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:29.070233 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.070031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw"
Jul 08 19:05:29.078303 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.078274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:29.083939 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.083915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t4wlb"
Jul 08 19:05:29.344688 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.344589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:29.344688 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.344646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:29.344688 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.344681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344768    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344805    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344826    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344839    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344839    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344855    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:30.34483334 +0000 UTC m=+4.009395873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344892    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:05:30.34487241 +0000 UTC m=+4.009434956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:29.344972 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.344913    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:30.344903096 +0000 UTC m=+4.009465628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:29.811301 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.811158    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:00:28 +0000 UTC" deadline="2028-01-27 01:03:09.505610563 +0000 UTC"
Jul 08 19:05:29.811301 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.811243    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13613h57m39.694372354s"
Jul 08 19:05:29.924000 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:29.923970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:29.924155 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:29.924077    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:29.931847 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:29.931629    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55a4251_70cc_42b2_bf2b_372748f73320.slice/crio-2732a1cb44582a1e799ccc768d002fe8b17418305874244453658f805255cfc4 WatchSource:0}: Error finding container 2732a1cb44582a1e799ccc768d002fe8b17418305874244453658f805255cfc4: Status 404 returned error can't find the container with id 2732a1cb44582a1e799ccc768d002fe8b17418305874244453658f805255cfc4
Jul 08 19:05:29.933716 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:29.933689    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c3b6fe_289b_427e_bb03_831dae993b10.slice/crio-152bf8686a08024d6f6d5392da2341ee0b7fee08b0c7a2fc4368fc9094765cd7 WatchSource:0}: Error finding container 152bf8686a08024d6f6d5392da2341ee0b7fee08b0c7a2fc4368fc9094765cd7: Status 404 returned error can't find the container with id 152bf8686a08024d6f6d5392da2341ee0b7fee08b0c7a2fc4368fc9094765cd7
Jul 08 19:05:29.937753 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:29.936440    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb99271_9db5_4d6a_b3a9_152683b48ca5.slice/crio-c53d0ccc967e964ef6a561b7741b05e0b2ec0c284f6dfdb0872dfd092bcb8b71 WatchSource:0}: Error finding container c53d0ccc967e964ef6a561b7741b05e0b2ec0c284f6dfdb0872dfd092bcb8b71: Status 404 returned error can't find the container with id c53d0ccc967e964ef6a561b7741b05e0b2ec0c284f6dfdb0872dfd092bcb8b71
Jul 08 19:05:29.945360 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:29.944473    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d05400_7a62_4e6e_bb7b_97d3c4eaff92.slice/crio-271f95e9fd8a470e61e9811b275625f9be81bf16d0e247cf072316eb1ca62af5 WatchSource:0}: Error finding container 271f95e9fd8a470e61e9811b275625f9be81bf16d0e247cf072316eb1ca62af5: Status 404 returned error can't find the container with id 271f95e9fd8a470e61e9811b275625f9be81bf16d0e247cf072316eb1ca62af5
Jul 08 19:05:29.945360 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:29.944816    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b1cfea_7567_45ce_a015_d1947a311e07.slice/crio-d81f5aea1a91c6928f87b682a30924ca2feb628b0c10aae33c5c4e694e223b05 WatchSource:0}: Error finding container d81f5aea1a91c6928f87b682a30924ca2feb628b0c10aae33c5c4e694e223b05: Status 404 returned error can't find the container with id d81f5aea1a91c6928f87b682a30924ca2feb628b0c10aae33c5c4e694e223b05
Jul 08 19:05:29.946644 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:05:29.946597    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1677a6e8_b5a9_42a2_8a07_ce4b33ed3328.slice/crio-2de124ec50b8f79cd66eb2e36a1ff8b64e0e27dae76b038b7cb8c32b9736b718 WatchSource:0}: Error finding container 2de124ec50b8f79cd66eb2e36a1ff8b64e0e27dae76b038b7cb8c32b9736b718: Status 404 returned error can't find the container with id 2de124ec50b8f79cd66eb2e36a1ff8b64e0e27dae76b038b7cb8c32b9736b718
Jul 08 19:05:30.351470 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.351435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:30.351638 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.351497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:30.351638 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.351537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:30.351638 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351576    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:30.351638 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351620    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:30.351638 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351637    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:05:30.351849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351649    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:32.351630349 +0000 UTC m=+6.016192878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:30.351849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351651    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:05:30.351849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351666    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:30.351849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351674    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:32.35165915 +0000 UTC m=+6.016221700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:30.351849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.351703    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:05:32.351692464 +0000 UTC m=+6.016254995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:30.812029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.811983    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:00:28 +0000 UTC" deadline="2028-02-26 08:50:16.737640667 +0000 UTC"
Jul 08 19:05:30.812029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.812026    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14341h44m45.925619286s"
Jul 08 19:05:30.924634 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.924599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:30.924812 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.924749    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:30.925179 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.925158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:30.925277 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:30.925258    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:30.955563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.955518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t4wlb" event={"ID":"4fb99271-9db5-4d6a-b3a9-152683b48ca5","Type":"ContainerStarted","Data":"c53d0ccc967e964ef6a561b7741b05e0b2ec0c284f6dfdb0872dfd092bcb8b71"}
Jul 08 19:05:30.957728 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.957610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g2rm5" event={"ID":"11b1cfea-7567-45ce-a015-d1947a311e07","Type":"ContainerStarted","Data":"d81f5aea1a91c6928f87b682a30924ca2feb628b0c10aae33c5c4e694e223b05"}
Jul 08 19:05:30.960326 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.960291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw" event={"ID":"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92","Type":"ContainerStarted","Data":"271f95e9fd8a470e61e9811b275625f9be81bf16d0e247cf072316eb1ca62af5"}
Jul 08 19:05:30.971421 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.971382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wtcj7" event={"ID":"41698256-87d5-433e-9e4f-a3f7de1fbfc3","Type":"ContainerStarted","Data":"803c786e65736c7d5bf3bfa49763962433b1af77d3e62c8dcaac2bc760950e7b"}
Jul 08 19:05:30.981474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.981437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"152bf8686a08024d6f6d5392da2341ee0b7fee08b0c7a2fc4368fc9094765cd7"}
Jul 08 19:05:30.984420 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.984385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ntjss" event={"ID":"e55a4251-70cc-42b2-bf2b-372748f73320","Type":"ContainerStarted","Data":"2732a1cb44582a1e799ccc768d002fe8b17418305874244453658f805255cfc4"}
Jul 08 19:05:30.993635 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:30.993598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal" event={"ID":"991d6b5a2a481334cc1eaf58c5d1d435","Type":"ContainerStarted","Data":"48ea8da5c4c33ee8f008d0876c08f3c57107c4277a2fae55b7ee700a710bf0c4"}
Jul 08 19:05:31.001619 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.001581    2563 generic.go:356] "Generic (PLEG): container finished" podID="0213be8aee70fe73b207187d672b3779" containerID="e7e0efc7fcd9224959bb0a28b58b25e6a4cd879f7f7676b9ad9ec1c726a45a80" exitCode=0
Jul 08 19:05:31.001798 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.001685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal" event={"ID":"0213be8aee70fe73b207187d672b3779","Type":"ContainerDied","Data":"e7e0efc7fcd9224959bb0a28b58b25e6a4cd879f7f7676b9ad9ec1c726a45a80"}
Jul 08 19:05:31.011398 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.011359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerStarted","Data":"2de124ec50b8f79cd66eb2e36a1ff8b64e0e27dae76b038b7cb8c32b9736b718"}
Jul 08 19:05:31.022363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.022280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jxqwh" event={"ID":"71b5f67b-07b0-4a25-80aa-d196ac472005","Type":"ContainerStarted","Data":"31ca5a685103ed3ee4dd0f14ed2cc8dbe2b500bfc3b6ce1c43f97670df6e0a93"}
Jul 08 19:05:31.032241 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.032160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f2mkg" event={"ID":"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb","Type":"ContainerStarted","Data":"8cebcf63d80e3066f17100e69fb4ce17584703924569a46665d42c6f52744ed9"}
Jul 08 19:05:31.036066 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.036006    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-212.ec2.internal" podStartSLOduration=4.035986441 podStartE2EDuration="4.035986441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:31.014156643 +0000 UTC m=+4.678719196" watchObservedRunningTime="2026-07-08 19:05:31.035986441 +0000 UTC m=+4.700548993"
Jul 08 19:05:31.924422 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:31.924385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:31.924950 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:31.924518    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:32.044327 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:32.044238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal" event={"ID":"0213be8aee70fe73b207187d672b3779","Type":"ContainerStarted","Data":"e160243a79b26124d902847213dca8a52d96293ade4598349b60d3a6807ca130"}
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:32.367306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:32.367367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:32.367419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367542    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367600    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:36.367581869 +0000 UTC m=+10.032144421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367775    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367795    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367807    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367847    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:05:36.367834939 +0000 UTC m=+10.032397468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367906    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:32.369212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.367939    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:36.367928394 +0000 UTC m=+10.032490927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:32.925273 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:32.925239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:32.925739 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:32.925268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:32.925739 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.925379    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:32.925739 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:32.925438    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:33.924439 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:33.924409    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:33.924584 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:33.924505    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:34.924151 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:34.924031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:34.924542 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:34.924196    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:34.924669 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:34.924650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:34.924767 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:34.924750    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:35.924192 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:35.924157    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:35.924544 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:35.924293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:36.403930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:36.403992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:36.404046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404107    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404183    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:44.404162728 +0000 UTC m=+18.068725273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404189    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404209    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404222    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404267    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:05:44.404254514 +0000 UTC m=+18.068817062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404334    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:36.404431 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.404363    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:44.404354626 +0000 UTC m=+18.068917193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:36.925380 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:36.925341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:36.925827 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.925471    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:36.926115 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:36.925949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:36.926115 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:36.926052    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:37.924560 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:37.924526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:37.924727 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:37.924663    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:38.927559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:38.927529    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:38.927994 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:38.927653    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:38.927994 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:38.927702    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:38.927994 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:38.927802    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:39.924402 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:39.924363    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:39.924564 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:39.924496    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:40.924078 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:40.924043    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:40.924455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:40.924043    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:40.924455 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:40.924184    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:40.924455 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:40.924246    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:41.924051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:41.924015    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:41.924279 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:41.924170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:42.924611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:42.924570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:42.924611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:42.924618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:42.925142 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:42.924715    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:42.925142 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:42.924839    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:43.924596 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:43.924553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:43.924811 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:43.924697    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:44.466866 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:44.466827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:44.466866 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:44.466871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:44.466914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467001    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467044    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467048    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467059    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467071    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:44.467112 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467077    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:00.46705813 +0000 UTC m=+34.131620679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:05:44.467363 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467124    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:00.467105359 +0000 UTC m=+34.131667907 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:05:44.467363 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.467152    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:06:00.467140508 +0000 UTC m=+34.131703040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:05:44.924304 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:44.924273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:44.924453 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.924380    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:44.924532 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:44.924465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:44.924647 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:44.924615    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:45.924108 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:45.924050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:45.924592 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:45.924177    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:46.924916 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:46.924872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:46.925414 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:46.925014    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:46.925414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:46.925070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:46.925414 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:46.925211    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:47.924835 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:47.924791    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:47.924995 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:47.924954    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:48.924335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:48.924300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:48.924723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:48.924301    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:48.924723 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:48.924428    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:48.924723 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:48.924488    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:49.924430 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:49.924116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:49.925215 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:49.924515    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:50.087201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.087112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw" event={"ID":"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92","Type":"ContainerStarted","Data":"cf06ae60f1ce704587a6b6989a63f282b4b4723983d204514eeae663aa5a0f48"}
Jul 08 19:05:50.088831 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.088801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wtcj7" event={"ID":"41698256-87d5-433e-9e4f-a3f7de1fbfc3","Type":"ContainerStarted","Data":"7547d5931552b8bf36eb7872185c4d7b53df37d7ce77c96426b2ab8f0b7aaaf4"}
Jul 08 19:05:50.092547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.092517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"d07dca9bd28b44dd8fe4a94a4690430d59e8082b1b897e1b858ca0356429e992"}
Jul 08 19:05:50.092721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.092603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"1b6b7f3b7e9d564a14d9c55f97b41459fa7e0cb59dff450de589c7a2af5ef800"}
Jul 08 19:05:50.092721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.092650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"dc006856e8885939c37b3c78ab6fd6c1e7d12654ccb7d3b531d6dcbedda98ff1"}
Jul 08 19:05:50.092721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.092694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"ecad57043992da81b76aa657cf1b88bf2d11096b4ee0fbae95ca76fd662ffee2"}
Jul 08 19:05:50.092873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.092738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"613a078d051589a2dabc3ce3c15ce4a1a848820e3aef21822b78baa55612b5ab"}
Jul 08 19:05:50.092873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.092785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"4d397949b538bfea60e7dbb2836e60cd0e95bea9485a0f33ff77a885bca0c3fe"}
Jul 08 19:05:50.094329 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.094298    2563 generic.go:356] "Generic (PLEG): container finished" podID="1677a6e8-b5a9-42a2-8a07-ce4b33ed3328" containerID="65f5d74401472a397da433e8e7de248a4587b0f213ccc255fe1f6ef3edb66cbd" exitCode=0
Jul 08 19:05:50.094463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.094388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerDied","Data":"65f5d74401472a397da433e8e7de248a4587b0f213ccc255fe1f6ef3edb66cbd"}
Jul 08 19:05:50.096172 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.096101    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jxqwh" event={"ID":"71b5f67b-07b0-4a25-80aa-d196ac472005","Type":"ContainerStarted","Data":"d6da99ceb0f08f1d0036df1cf365554a55d9ec8578b2c02dda7e53e87febe9cd"}
Jul 08 19:05:50.097682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.097644    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f2mkg" event={"ID":"8b2f1675-9c4c-40dc-893e-6dd1ed09aecb","Type":"ContainerStarted","Data":"eedbca18696cb6aee8f81461432b5b185eb3b2ebb804cd1f8018c763450129c3"}
Jul 08 19:05:50.099539 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.099513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t4wlb" event={"ID":"4fb99271-9db5-4d6a-b3a9-152683b48ca5","Type":"ContainerStarted","Data":"35997f34fedfe9be24bededc4f1181f97bfd80d73b09003bd1d384c157aba7ac"}
Jul 08 19:05:50.100768 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.100743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g2rm5" event={"ID":"11b1cfea-7567-45ce-a015-d1947a311e07","Type":"ContainerStarted","Data":"3422abc1673ff8b72e11602601644680888d5ffbb514f6fb5723b7f2aef52817"}
Jul 08 19:05:50.129395 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.129342    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-212.ec2.internal" podStartSLOduration=23.129325403 podStartE2EDuration="23.129325403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:32.064530865 +0000 UTC m=+5.729093417" watchObservedRunningTime="2026-07-08 19:05:50.129325403 +0000 UTC m=+23.793887954"
Jul 08 19:05:50.162331 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.162279    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wtcj7" podStartSLOduration=4.847052713 podStartE2EDuration="24.162262085s" totalImagesPullingTime="19.315209372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:50.129980466 +0000 UTC m=+23.794543016" watchObservedRunningTime="2026-07-08 19:05:50.162262085 +0000 UTC m=+23.826824634"
Jul 08 19:05:50.209005 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.208947    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jxqwh" podStartSLOduration=4.955439558 podStartE2EDuration="24.208926985s" totalImagesPullingTime="19.253487427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:50.162953468 +0000 UTC m=+23.827516020" watchObservedRunningTime="2026-07-08 19:05:50.208926985 +0000 UTC m=+23.873489537"
Jul 08 19:05:50.277699 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.277653    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g2rm5" podStartSLOduration=5.02622534 podStartE2EDuration="24.277636707s" totalImagesPullingTime="19.251411367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:50.276925835 +0000 UTC m=+23.941488400" watchObservedRunningTime="2026-07-08 19:05:50.277636707 +0000 UTC m=+23.942199259"
Jul 08 19:05:50.411274 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.411219    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t4wlb" podStartSLOduration=5.154137508 podStartE2EDuration="24.411200855s" totalImagesPullingTime="19.257063347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:50.402632345 +0000 UTC m=+24.067194896" watchObservedRunningTime="2026-07-08 19:05:50.411200855 +0000 UTC m=+24.075763406"
Jul 08 19:05:50.411534 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.411487    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f2mkg" podStartSLOduration=5.155516203 podStartE2EDuration="24.411482465s" totalImagesPullingTime="19.255966262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:50.371652092 +0000 UTC m=+24.036214644" watchObservedRunningTime="2026-07-08 19:05:50.411482465 +0000 UTC m=+24.076045016"
Jul 08 19:05:50.819558 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.819525    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 19:05:50.924260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.924181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:50.924432 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:50.924179    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:50.924432 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:50.924329    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:50.924432 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:50.924394    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:51.104557 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.104519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw" event={"ID":"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92","Type":"ContainerStarted","Data":"0a8e0d6a64ed34695ccebacf896b08957ff7fdbfa6dd0ce91b1eeb8eba204599"}
Jul 08 19:05:51.105915 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.105886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ntjss" event={"ID":"e55a4251-70cc-42b2-bf2b-372748f73320","Type":"ContainerStarted","Data":"10fb56442ae90047f60a9a8c095210d5c183c8a725f12bb2109f9f0a5c125ffc"}
Jul 08 19:05:51.134488 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.134433    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ntjss" podStartSLOduration=5.862664087 podStartE2EDuration="25.134416967s" totalImagesPullingTime="19.27175288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:51.134264598 +0000 UTC m=+24.798827150" watchObservedRunningTime="2026-07-08 19:05:51.134416967 +0000 UTC m=+24.798979518"
Jul 08 19:05:51.688476 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.688441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:51.689151 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.689127    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:51.795844 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.795726    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:05:50.819550582Z","UUID":"e4624f4d-27b6-4267-916b-487b63fda2e5","Handler":null,"Name":"","Endpoint":""}
Jul 08 19:05:51.797490 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.797463    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 19:05:51.797628 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.797513    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 19:05:51.924007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:51.923970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:51.924212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:51.924138    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:52.112389 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:52.112193    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:52.112708 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:52.112626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f2mkg"
Jul 08 19:05:52.924119 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:52.924067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:52.924314 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:52.924124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:52.924314 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:52.924217    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:52.924433 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:52.924346    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:53.116967 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:53.116924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw" event={"ID":"e2d05400-7a62-4e6e-bb7b-97d3c4eaff92","Type":"ContainerStarted","Data":"bb0b69f0185f0b0cc73be25fbd201ffe4923050028a052396262fcea9f0a32b3"}
Jul 08 19:05:53.120718 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:53.120682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"eb425aee2e4ca115df718a4a34a7659cd23878912d02538e1e315d3a697eaaec"}
Jul 08 19:05:53.923812 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:53.923783    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:53.924021 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:53.923919    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:54.924188 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:54.924152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:54.924801 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:54.924301    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:54.924801 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:54.924368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:54.924801 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:54.924493    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:55.924262 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:55.923991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:55.924762 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:55.924273    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:56.130404 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.130358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerStarted","Data":"fcb8dd48fdee83aa100c12f38367388aa12848b3ed172c512aeda8a55f429fde"}
Jul 08 19:05:56.134011 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.133980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" event={"ID":"a9c3b6fe-289b-427e-bb03-831dae993b10","Type":"ContainerStarted","Data":"2e66d399a0211ba1c3f3d2f25398f90c9c173a779a500cc429eaf9c2a47c038b"}
Jul 08 19:05:56.134287 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.134260    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:56.134392 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.134320    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:56.151332 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.151305    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:56.161873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.161829    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7btw" podStartSLOduration=8.509358825 podStartE2EDuration="30.161815268s" totalImagesPullingTime="21.652456443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:53.14771549 +0000 UTC m=+26.812278040" watchObservedRunningTime="2026-07-08 19:05:56.161815268 +0000 UTC m=+29.826377818"
Jul 08 19:05:56.927796 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.926310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:56.927796 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:56.926459    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:56.927796 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:56.926907    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:56.927796 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:56.927040    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:57.137847 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.137814    2563 generic.go:356] "Generic (PLEG): container finished" podID="1677a6e8-b5a9-42a2-8a07-ce4b33ed3328" containerID="fcb8dd48fdee83aa100c12f38367388aa12848b3ed172c512aeda8a55f429fde" exitCode=0
Jul 08 19:05:57.138022 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.137901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerDied","Data":"fcb8dd48fdee83aa100c12f38367388aa12848b3ed172c512aeda8a55f429fde"}
Jul 08 19:05:57.138352 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.138335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:57.155584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.155551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:05:57.174434 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.174388    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv" podStartSLOduration=11.838635694 podStartE2EDuration="31.174374071s" totalImagesPullingTime="19.335738377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:56.202317121 +0000 UTC m=+29.866879673" watchObservedRunningTime="2026-07-08 19:05:57.174374071 +0000 UTC m=+30.838936631"
Jul 08 19:05:57.831184 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.831153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kxb2x"]
Jul 08 19:05:57.831328 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.831278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:57.831417 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:57.831379    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:57.838174 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.838144    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vrp9v"]
Jul 08 19:05:57.838343 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.838257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:57.838385 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:57.838354    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:05:57.838622 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.838601    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4sk5h"]
Jul 08 19:05:57.838684 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:57.838674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:57.838774 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:57.838758    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:58.924046 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:58.923854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:05:58.924702 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:58.924163    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:05:59.144628 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:59.144592    2563 generic.go:356] "Generic (PLEG): container finished" podID="1677a6e8-b5a9-42a2-8a07-ce4b33ed3328" containerID="0c20333420b52de6f03eeb14df6346ee066529f77433432d7c97934e2fb0a3f0" exitCode=0
Jul 08 19:05:59.144768 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:59.144687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerDied","Data":"0c20333420b52de6f03eeb14df6346ee066529f77433432d7c97934e2fb0a3f0"}
Jul 08 19:05:59.923985 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:59.923950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:05:59.924157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:05:59.923958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:05:59.924157 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:59.924067    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:05:59.924549 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:05:59.924150    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:06:00.149555 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:00.149508    2563 generic.go:356] "Generic (PLEG): container finished" podID="1677a6e8-b5a9-42a2-8a07-ce4b33ed3328" containerID="16f81cebbc4cb0d93475638c2eefabcf4c91fb8db71ccff8314bafe74bd59c9a" exitCode=0
Jul 08 19:06:00.149704 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:00.149575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerDied","Data":"16f81cebbc4cb0d93475638c2eefabcf4c91fb8db71ccff8314bafe74bd59c9a"}
Jul 08 19:06:00.483044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:00.482952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:06:00.483044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:00.483001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:00.483044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:00.483032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483116    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483120    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483178    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret podName:bcc2336f-9b1b-401d-89a6-dd190e97c7b5 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:32.483165075 +0000 UTC m=+66.147727617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret") pod "global-pull-secret-syncer-4sk5h" (UID: "bcc2336f-9b1b-401d-89a6-dd190e97c7b5") : object "kube-system"/"original-pull-secret" not registered
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483189    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483216    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483229    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cml54 for pod openshift-network-diagnostics/network-check-target-vrp9v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483191    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs podName:69ca6196-d827-45d8-985d-b0d8ffcd0f57 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:32.483184543 +0000 UTC m=+66.147747073 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs") pod "network-metrics-daemon-kxb2x" (UID: "69ca6196-d827-45d8-985d-b0d8ffcd0f57") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 19:06:00.483346 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.483288    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54 podName:7a0f6079-82a2-4e15-b3a4-786abf7ad66e nodeName:}" failed. No retries permitted until 2026-07-08 19:06:32.483270923 +0000 UTC m=+66.147833452 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cml54" (UniqueName: "kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54") pod "network-check-target-vrp9v" (UID: "7a0f6079-82a2-4e15-b3a4-786abf7ad66e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 19:06:00.924528 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:00.924495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:06:00.925065 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:00.924625    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4sk5h" podUID="bcc2336f-9b1b-401d-89a6-dd190e97c7b5"
Jul 08 19:06:01.923880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:01.923839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:06:01.923880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:01.923879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:01.924160 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:01.923975    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kxb2x" podUID="69ca6196-d827-45d8-985d-b0d8ffcd0f57"
Jul 08 19:06:01.924160 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:01.924071    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrp9v" podUID="7a0f6079-82a2-4e15-b3a4-786abf7ad66e"
Jul 08 19:06:02.118471 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.118438    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-212.ec2.internal" event="NodeReady"
Jul 08 19:06:02.119048 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.118588    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 19:06:02.171860 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.171765    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zfw8l"]
Jul 08 19:06:02.173877 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.173853    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2k28s"]
Jul 08 19:06:02.174016 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.173984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.175883 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.175859    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.176619 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.176599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 19:06:02.176735 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.176623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 19:06:02.176735 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.176658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxdf4\""
Jul 08 19:06:02.178202 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.178181    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qnk2b\""
Jul 08 19:06:02.178368 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.178271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 19:06:02.178551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.178529    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 19:06:02.178551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.178550    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 19:06:02.182120 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.182037    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2k28s"]
Jul 08 19:06:02.185920 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.185896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zfw8l"]
Jul 08 19:06:02.295016 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.294980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.295227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.295056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b806e0a5-276a-4d25-93ec-e909b812ab48-tmp-dir\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.295227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.295108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4l82\" (UniqueName: \"kubernetes.io/projected/b806e0a5-276a-4d25-93ec-e909b812ab48-kube-api-access-j4l82\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.295227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.295159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b806e0a5-276a-4d25-93ec-e909b812ab48-config-volume\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.295227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.295179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35c3e76c-462d-4501-a8b7-e6483368606a-cert\") pod \"ingress-canary-2k28s\" (UID: \"35c3e76c-462d-4501-a8b7-e6483368606a\") " pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.295227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.295203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4bfc\" (UniqueName: \"kubernetes.io/projected/35c3e76c-462d-4501-a8b7-e6483368606a-kube-api-access-q4bfc\") pod \"ingress-canary-2k28s\" (UID: \"35c3e76c-462d-4501-a8b7-e6483368606a\") " pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.396382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b806e0a5-276a-4d25-93ec-e909b812ab48-config-volume\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.396382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35c3e76c-462d-4501-a8b7-e6483368606a-cert\") pod \"ingress-canary-2k28s\" (UID: \"35c3e76c-462d-4501-a8b7-e6483368606a\") " pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.396382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4bfc\" (UniqueName: \"kubernetes.io/projected/35c3e76c-462d-4501-a8b7-e6483368606a-kube-api-access-q4bfc\") pod \"ingress-canary-2k28s\" (UID: \"35c3e76c-462d-4501-a8b7-e6483368606a\") " pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.396652 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.396652 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b806e0a5-276a-4d25-93ec-e909b812ab48-tmp-dir\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.396652 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4l82\" (UniqueName: \"kubernetes.io/projected/b806e0a5-276a-4d25-93ec-e909b812ab48-kube-api-access-j4l82\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.396652 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:02.396535    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 19:06:02.396652 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:02.396616    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls podName:b806e0a5-276a-4d25-93ec-e909b812ab48 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:02.896593623 +0000 UTC m=+36.561156178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls") pod "dns-default-zfw8l" (UID: "b806e0a5-276a-4d25-93ec-e909b812ab48") : secret "dns-default-metrics-tls" not found
Jul 08 19:06:02.396881 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396794    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b806e0a5-276a-4d25-93ec-e909b812ab48-tmp-dir\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.396990 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.396971    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b806e0a5-276a-4d25-93ec-e909b812ab48-config-volume\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.400836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.400816    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35c3e76c-462d-4501-a8b7-e6483368606a-cert\") pod \"ingress-canary-2k28s\" (UID: \"35c3e76c-462d-4501-a8b7-e6483368606a\") " pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.406125 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.406073    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4l82\" (UniqueName: \"kubernetes.io/projected/b806e0a5-276a-4d25-93ec-e909b812ab48-kube-api-access-j4l82\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.406447 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.406428    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4bfc\" (UniqueName: \"kubernetes.io/projected/35c3e76c-462d-4501-a8b7-e6483368606a-kube-api-access-q4bfc\") pod \"ingress-canary-2k28s\" (UID: \"35c3e76c-462d-4501-a8b7-e6483368606a\") " pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.495526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.495489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2k28s"
Jul 08 19:06:02.645928 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.645717    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2k28s"]
Jul 08 19:06:02.649722 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:02.649687    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c3e76c_462d_4501_a8b7_e6483368606a.slice/crio-fb593ded86fd33aa59bd1c364ad1dc9140d0fbbafc16a8c78ced5a333aca2262 WatchSource:0}: Error finding container fb593ded86fd33aa59bd1c364ad1dc9140d0fbbafc16a8c78ced5a333aca2262: Status 404 returned error can't find the container with id fb593ded86fd33aa59bd1c364ad1dc9140d0fbbafc16a8c78ced5a333aca2262
Jul 08 19:06:02.900302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.900203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:02.900460 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:02.900368    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 19:06:02.900460 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:02.900451    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls podName:b806e0a5-276a-4d25-93ec-e909b812ab48 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:03.900426083 +0000 UTC m=+37.564988638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls") pod "dns-default-zfw8l" (UID: "b806e0a5-276a-4d25-93ec-e909b812ab48") : secret "dns-default-metrics-tls" not found
Jul 08 19:06:02.924643 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.924606    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:06:02.927877 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:02.927851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 19:06:03.162098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.161991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2k28s" event={"ID":"35c3e76c-462d-4501-a8b7-e6483368606a","Type":"ContainerStarted","Data":"fb593ded86fd33aa59bd1c364ad1dc9140d0fbbafc16a8c78ced5a333aca2262"}
Jul 08 19:06:03.909667 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.909624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:03.909855 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:03.909820    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 19:06:03.909922 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:03.909910    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls podName:b806e0a5-276a-4d25-93ec-e909b812ab48 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:05.909891591 +0000 UTC m=+39.574454140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls") pod "dns-default-zfw8l" (UID: "b806e0a5-276a-4d25-93ec-e909b812ab48") : secret "dns-default-metrics-tls" not found
Jul 08 19:06:03.923950 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.923913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:03.924172 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.924136    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:06:03.928232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.928203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 19:06:03.928232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.928231    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fsv9h\""
Jul 08 19:06:03.928427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.928236    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 19:06:03.928427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.928259    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 19:06:03.928624 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:03.928599    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfh9q\""
Jul 08 19:06:04.861912 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:04.861878    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"]
Jul 08 19:06:04.875692 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:04.875625    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"]
Jul 08 19:06:04.875692 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:04.875662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:04.879518 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:04.879487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 08 19:06:04.879681 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:04.879560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-96ml2\""
Jul 08 19:06:04.879862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:04.879552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 08 19:06:05.022072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.022032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:05.022253 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.022136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63deeaca-7675-41e6-b345-191f40b70d31-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:05.122591 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.122520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63deeaca-7675-41e6-b345-191f40b70d31-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:05.122764 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.122609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:05.122764 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:05.122728    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 19:06:05.122867 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:05.122801    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert podName:63deeaca-7675-41e6-b345-191f40b70d31 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:05.622783717 +0000 UTC m=+39.287346248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4flzn" (UID: "63deeaca-7675-41e6-b345-191f40b70d31") : secret "networking-console-plugin-cert" not found
Jul 08 19:06:05.123391 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.123363    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63deeaca-7675-41e6-b345-191f40b70d31-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:05.626231 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.626193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:05.626410 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:05.626350    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 19:06:05.626459 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:05.626426    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert podName:63deeaca-7675-41e6-b345-191f40b70d31 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:06.626409277 +0000 UTC m=+40.290971824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4flzn" (UID: "63deeaca-7675-41e6-b345-191f40b70d31") : secret "networking-console-plugin-cert" not found
Jul 08 19:06:05.928650 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:05.928567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:05.929010 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:05.928696    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 19:06:05.929010 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:05.928747    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls podName:b806e0a5-276a-4d25-93ec-e909b812ab48 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:09.928734431 +0000 UTC m=+43.593296960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls") pod "dns-default-zfw8l" (UID: "b806e0a5-276a-4d25-93ec-e909b812ab48") : secret "dns-default-metrics-tls" not found
Jul 08 19:06:06.632814 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:06.632774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:06.632982 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:06.632939    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 19:06:06.633019 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:06.633011    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert podName:63deeaca-7675-41e6-b345-191f40b70d31 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.632994481 +0000 UTC m=+42.297557009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4flzn" (UID: "63deeaca-7675-41e6-b345-191f40b70d31") : secret "networking-console-plugin-cert" not found
Jul 08 19:06:07.174494 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.174213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2k28s" event={"ID":"35c3e76c-462d-4501-a8b7-e6483368606a","Type":"ContainerStarted","Data":"33134b9508d61830273056cf26fb75eb97b8c260cf3b8cda30a844bf76853146"}
Jul 08 19:06:07.177024 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.176999    2563 generic.go:356] "Generic (PLEG): container finished" podID="1677a6e8-b5a9-42a2-8a07-ce4b33ed3328" containerID="293f5369674e0d935bb518d87a3dba031f4faaaa12b344cd86b9fb44525d8c63" exitCode=0
Jul 08 19:06:07.177179 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.177038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerDied","Data":"293f5369674e0d935bb518d87a3dba031f4faaaa12b344cd86b9fb44525d8c63"}
Jul 08 19:06:07.201277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.201228    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2k28s" podStartSLOduration=1.745657387 podStartE2EDuration="5.201212095s" totalImagesPullingTime="3.455554708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:07.200533834 +0000 UTC m=+40.865096396" watchObservedRunningTime="2026-07-08 19:06:07.201212095 +0000 UTC m=+40.865774645"
Jul 08 19:06:07.350688 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.350653    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qw687"]
Jul 08 19:06:07.356115 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.356064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.359017 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.358973    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 19:06:07.359017 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.358973    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 19:06:07.359349 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.359299    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vczqd\""
Jul 08 19:06:07.359426 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.359409    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 19:06:07.359707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.359678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 19:06:07.366895 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.366863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qw687"]
Jul 08 19:06:07.438890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.438801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.438890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.438849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96d21489-9846-4fac-aa04-91845fd9e46b-data-volume\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.438890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.438873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96d21489-9846-4fac-aa04-91845fd9e46b-crio-socket\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.439195 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.438980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.439195 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.439030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmggw\" (UniqueName: \"kubernetes.io/projected/96d21489-9846-4fac-aa04-91845fd9e46b-kube-api-access-wmggw\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.539715 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.539678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.539879 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.539726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96d21489-9846-4fac-aa04-91845fd9e46b-data-volume\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.539879 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:07.539832    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:07.539954 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:07.539899    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.039880919 +0000 UTC m=+41.704443447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:07.539954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.539917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96d21489-9846-4fac-aa04-91845fd9e46b-crio-socket\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.540034 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.539951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.540034 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.539988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmggw\" (UniqueName: \"kubernetes.io/projected/96d21489-9846-4fac-aa04-91845fd9e46b-kube-api-access-wmggw\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.540135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.540037    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96d21489-9846-4fac-aa04-91845fd9e46b-data-volume\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.540135 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:07.540043    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 19:06:07.540135 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:07.540120    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.040107654 +0000 UTC m=+41.704670198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : configmap "kube-rbac-proxy" not found
Jul 08 19:06:07.540288 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.540183    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96d21489-9846-4fac-aa04-91845fd9e46b-crio-socket\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:07.551632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:07.551598    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmggw\" (UniqueName: \"kubernetes.io/projected/96d21489-9846-4fac-aa04-91845fd9e46b-kube-api-access-wmggw\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:08.044201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:08.044156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:08.044359 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:08.044269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:08.044359 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:08.044287    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 19:06:08.044359 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:08.044352    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:09.044332658 +0000 UTC m=+42.708895204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : configmap "kube-rbac-proxy" not found
Jul 08 19:06:08.044479 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:08.044391    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:08.044479 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:08.044431    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:09.044420227 +0000 UTC m=+42.708982773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:08.182755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:08.182713    2563 generic.go:356] "Generic (PLEG): container finished" podID="1677a6e8-b5a9-42a2-8a07-ce4b33ed3328" containerID="a188f69d387c8f3589258a856986579dab9134bb907729f690e5f6e5acc5370f" exitCode=0
Jul 08 19:06:08.183149 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:08.182812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerDied","Data":"a188f69d387c8f3589258a856986579dab9134bb907729f690e5f6e5acc5370f"}
Jul 08 19:06:08.649415 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:08.649379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:08.649577 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:08.649533    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 19:06:08.649618 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:08.649601    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert podName:63deeaca-7675-41e6-b345-191f40b70d31 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:12.649585409 +0000 UTC m=+46.314147937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4flzn" (UID: "63deeaca-7675-41e6-b345-191f40b70d31") : secret "networking-console-plugin-cert" not found
Jul 08 19:06:09.053061 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:09.052953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:09.053255 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:09.053066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:09.053255 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:09.053121    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 19:06:09.053255 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:09.053206    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:11.053184471 +0000 UTC m=+44.717747017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : configmap "kube-rbac-proxy" not found
Jul 08 19:06:09.053255 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:09.053206    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:09.053255 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:09.053252    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:11.053241042 +0000 UTC m=+44.717803574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:09.164067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:09.164042    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g2rm5_11b1cfea-7567-45ce-a015-d1947a311e07/dns-node-resolver/0.log"
Jul 08 19:06:09.190616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:09.190582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" event={"ID":"1677a6e8-b5a9-42a2-8a07-ce4b33ed3328","Type":"ContainerStarted","Data":"c4d6c9986c5f1b9f61e501b48180203dedb544e8a1728c24c1d8a8d64bce5ec6"}
Jul 08 19:06:09.217927 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:09.217862    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gmmcp" podStartSLOduration=10.337805254 podStartE2EDuration="43.217845679s" totalImagesPullingTime="32.880040425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:09.216433748 +0000 UTC m=+42.880996298" watchObservedRunningTime="2026-07-08 19:06:09.217845679 +0000 UTC m=+42.882408229"
Jul 08 19:06:09.961575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:09.961534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:09.961751 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:09.961674    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 19:06:09.961751 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:09.961734    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls podName:b806e0a5-276a-4d25-93ec-e909b812ab48 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:17.961720496 +0000 UTC m=+51.626283037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls") pod "dns-default-zfw8l" (UID: "b806e0a5-276a-4d25-93ec-e909b812ab48") : secret "dns-default-metrics-tls" not found
Jul 08 19:06:10.330293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.330208    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xzzzk"]
Jul 08 19:06:10.349331 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.349304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xzzzk"]
Jul 08 19:06:10.349482 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.349401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.352277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.352244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 08 19:06:10.353315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.353289    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 08 19:06:10.353315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.353304    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 08 19:06:10.353455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.353302    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vnt79\""
Jul 08 19:06:10.353455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.353301    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 08 19:06:10.360979 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.360956    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t4wlb_4fb99271-9db5-4d6a-b3a9-152683b48ca5/node-ca/0.log"
Jul 08 19:06:10.465208 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.465169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d250a92a-bc53-4459-935d-5e1b88899ece-signing-cabundle\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.465366 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.465231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d250a92a-bc53-4459-935d-5e1b88899ece-signing-key\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.465366 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.465315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spmxv\" (UniqueName: \"kubernetes.io/projected/d250a92a-bc53-4459-935d-5e1b88899ece-kube-api-access-spmxv\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.566387 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.566351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d250a92a-bc53-4459-935d-5e1b88899ece-signing-key\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.566519 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.566413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spmxv\" (UniqueName: \"kubernetes.io/projected/d250a92a-bc53-4459-935d-5e1b88899ece-kube-api-access-spmxv\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.566519 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.566464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d250a92a-bc53-4459-935d-5e1b88899ece-signing-cabundle\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.567221 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.567118    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d250a92a-bc53-4459-935d-5e1b88899ece-signing-cabundle\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.568850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.568829    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d250a92a-bc53-4459-935d-5e1b88899ece-signing-key\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.576418 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.576394    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spmxv\" (UniqueName: \"kubernetes.io/projected/d250a92a-bc53-4459-935d-5e1b88899ece-kube-api-access-spmxv\") pod \"service-ca-758f599c5-xzzzk\" (UID: \"d250a92a-bc53-4459-935d-5e1b88899ece\") " pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.658537 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.658507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xzzzk"
Jul 08 19:06:10.807588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.807556    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xzzzk"]
Jul 08 19:06:10.810280 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:10.810248    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd250a92a_bc53_4459_935d_5e1b88899ece.slice/crio-af88426250536d0e27035fb7c7a32528cca4421cb00a9d355247e843a3ddb756 WatchSource:0}: Error finding container af88426250536d0e27035fb7c7a32528cca4421cb00a9d355247e843a3ddb756: Status 404 returned error can't find the container with id af88426250536d0e27035fb7c7a32528cca4421cb00a9d355247e843a3ddb756
Jul 08 19:06:10.960766 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:10.960662    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2k28s_35c3e76c-462d-4501-a8b7-e6483368606a/serve-healthcheck-canary/0.log"
Jul 08 19:06:11.070596 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:11.070556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:11.070777 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:11.070650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:11.070777 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:11.070692    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 19:06:11.070777 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:11.070747    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:11.070965 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:11.070783    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:15.070754245 +0000 UTC m=+48.735316775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : configmap "kube-rbac-proxy" not found
Jul 08 19:06:11.070965 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:11.070805    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:15.070791814 +0000 UTC m=+48.735354343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:11.197343 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:11.197303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xzzzk" event={"ID":"d250a92a-bc53-4459-935d-5e1b88899ece","Type":"ContainerStarted","Data":"af88426250536d0e27035fb7c7a32528cca4421cb00a9d355247e843a3ddb756"}
Jul 08 19:06:12.685373 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:12.685335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:12.685742 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:12.685514    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 19:06:12.685742 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:12.685590    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert podName:63deeaca-7675-41e6-b345-191f40b70d31 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:20.685574841 +0000 UTC m=+54.350137401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4flzn" (UID: "63deeaca-7675-41e6-b345-191f40b70d31") : secret "networking-console-plugin-cert" not found
Jul 08 19:06:14.207907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:14.207869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xzzzk" event={"ID":"d250a92a-bc53-4459-935d-5e1b88899ece","Type":"ContainerStarted","Data":"88a9341510f0f2279c74e6c2bea0c27fe4446dab25554876dc1708f151744598"}
Jul 08 19:06:14.225169 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:14.225110    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xzzzk" podStartSLOduration=1.247339701 podStartE2EDuration="4.225070919s" totalImagesPullingTime="2.977731218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:14.224419712 +0000 UTC m=+47.888982264" watchObservedRunningTime="2026-07-08 19:06:14.225070919 +0000 UTC m=+47.889633470"
Jul 08 19:06:15.106098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:15.106036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:15.106300 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:15.106142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:15.106300 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:15.106222    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:15.106387 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:15.106306    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:23.106285998 +0000 UTC m=+56.770848542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:15.106667 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:15.106232    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 19:06:15.106711 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:15.106701    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:23.10668783 +0000 UTC m=+56.771250377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : configmap "kube-rbac-proxy" not found
Jul 08 19:06:18.030682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:18.030641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:18.031278 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:18.030812    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 19:06:18.031278 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:18.030890    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls podName:b806e0a5-276a-4d25-93ec-e909b812ab48 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:34.030870333 +0000 UTC m=+67.695432867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls") pod "dns-default-zfw8l" (UID: "b806e0a5-276a-4d25-93ec-e909b812ab48") : secret "dns-default-metrics-tls" not found
Jul 08 19:06:20.749917 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:20.749878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:20.750418 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:20.750061    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 19:06:20.750418 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:20.750165    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert podName:63deeaca-7675-41e6-b345-191f40b70d31 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:36.750146659 +0000 UTC m=+70.414709188 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4flzn" (UID: "63deeaca-7675-41e6-b345-191f40b70d31") : secret "networking-console-plugin-cert" not found
Jul 08 19:06:23.167456 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:23.167412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:23.167885 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:23.167496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:23.167885 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:23.167589    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:23.167885 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:23.167675    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls podName:96d21489-9846-4fac-aa04-91845fd9e46b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:39.167638307 +0000 UTC m=+72.832200836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qw687" (UID: "96d21489-9846-4fac-aa04-91845fd9e46b") : secret "insights-runtime-extractor-tls" not found
Jul 08 19:06:23.167992 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:23.167933    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96d21489-9846-4fac-aa04-91845fd9e46b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:29.156132 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:29.156103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvxjv"
Jul 08 19:06:31.622806 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.622765    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8668666774-m8zdk"]
Jul 08 19:06:31.625766 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.625749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.628376 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.628341    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 19:06:31.628376 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.628356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 19:06:31.628570 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.628404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 19:06:31.629584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.629562    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mxm59\""
Jul 08 19:06:31.637068 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.637044    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 19:06:31.655980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.655950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8668666774-m8zdk"]
Jul 08 19:06:31.729303 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16289e5d-6029-465a-b20d-7138d7ab4206-installation-pull-secrets\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729471 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16289e5d-6029-465a-b20d-7138d7ab4206-registry-certificates\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729471 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16289e5d-6029-465a-b20d-7138d7ab4206-image-registry-private-configuration\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729471 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16289e5d-6029-465a-b20d-7138d7ab4206-trusted-ca\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729471 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-registry-tls\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729601 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-bound-sa-token\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729601 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbkjw\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-kube-api-access-gbkjw\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.729663 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.729577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16289e5d-6029-465a-b20d-7138d7ab4206-ca-trust-extracted\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830348 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-bound-sa-token\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830348 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbkjw\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-kube-api-access-gbkjw\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830610 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16289e5d-6029-465a-b20d-7138d7ab4206-ca-trust-extracted\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830610 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16289e5d-6029-465a-b20d-7138d7ab4206-installation-pull-secrets\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16289e5d-6029-465a-b20d-7138d7ab4206-registry-certificates\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16289e5d-6029-465a-b20d-7138d7ab4206-image-registry-private-configuration\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16289e5d-6029-465a-b20d-7138d7ab4206-trusted-ca\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.830792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.830714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-registry-tls\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.831053 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.831030    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16289e5d-6029-465a-b20d-7138d7ab4206-ca-trust-extracted\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.831577 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.831549    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16289e5d-6029-465a-b20d-7138d7ab4206-registry-certificates\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.831687 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.831625    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16289e5d-6029-465a-b20d-7138d7ab4206-trusted-ca\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.833461 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.833434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16289e5d-6029-465a-b20d-7138d7ab4206-installation-pull-secrets\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.833540 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.833480    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16289e5d-6029-465a-b20d-7138d7ab4206-image-registry-private-configuration\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.833606 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.833590    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-registry-tls\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.840709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.840680    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbkjw\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-kube-api-access-gbkjw\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.864062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.864034    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16289e5d-6029-465a-b20d-7138d7ab4206-bound-sa-token\") pod \"image-registry-8668666774-m8zdk\" (UID: \"16289e5d-6029-465a-b20d-7138d7ab4206\") " pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:31.936730 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:31.936628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:32.109864 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.109832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8668666774-m8zdk"]
Jul 08 19:06:32.112951 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:32.112911    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16289e5d_6029_465a_b20d_7138d7ab4206.slice/crio-2d6d06aebde2269366ccb0e762650cefea8496e1f3ae8f8d504b4f5956c6d96d WatchSource:0}: Error finding container 2d6d06aebde2269366ccb0e762650cefea8496e1f3ae8f8d504b4f5956c6d96d: Status 404 returned error can't find the container with id 2d6d06aebde2269366ccb0e762650cefea8496e1f3ae8f8d504b4f5956c6d96d
Jul 08 19:06:32.253067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.252983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8668666774-m8zdk" event={"ID":"16289e5d-6029-465a-b20d-7138d7ab4206","Type":"ContainerStarted","Data":"fdc16919a19f8ae592b5624df88cbbfe9824e0588a290e0fa7269a9d672fdf9f"}
Jul 08 19:06:32.253067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.253031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8668666774-m8zdk" event={"ID":"16289e5d-6029-465a-b20d-7138d7ab4206","Type":"ContainerStarted","Data":"2d6d06aebde2269366ccb0e762650cefea8496e1f3ae8f8d504b4f5956c6d96d"}
Jul 08 19:06:32.253303 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.253117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:32.302256 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.302200    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8668666774-m8zdk" podStartSLOduration=1.302183988 podStartE2EDuration="1.302183988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:32.300146212 +0000 UTC m=+65.964708763" watchObservedRunningTime="2026-07-08 19:06:32.302183988 +0000 UTC m=+65.966746538"
Jul 08 19:06:32.537259 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.537161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:32.537428 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.537269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:06:32.537428 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.537331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:06:32.540217 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.540192    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 19:06:32.540897 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.540877    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 19:06:32.540974 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.540912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 19:06:32.550535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.550505    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69ca6196-d827-45d8-985d-b0d8ffcd0f57-metrics-certs\") pod \"network-metrics-daemon-kxb2x\" (UID: \"69ca6196-d827-45d8-985d-b0d8ffcd0f57\") " pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:06:32.550668 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.550597    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcc2336f-9b1b-401d-89a6-dd190e97c7b5-original-pull-secret\") pod \"global-pull-secret-syncer-4sk5h\" (UID: \"bcc2336f-9b1b-401d-89a6-dd190e97c7b5\") " pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:06:32.550728 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.550705    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 19:06:32.561660 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.561634    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cml54\" (UniqueName: \"kubernetes.io/projected/7a0f6079-82a2-4e15-b3a4-786abf7ad66e-kube-api-access-cml54\") pod \"network-check-target-vrp9v\" (UID: \"7a0f6079-82a2-4e15-b3a4-786abf7ad66e\") " pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:32.636826 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.636789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4sk5h"
Jul 08 19:06:32.742250 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.742218    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fsv9h\""
Jul 08 19:06:32.748723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.748693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfh9q\""
Jul 08 19:06:32.749707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.749689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:32.756926 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.756888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kxb2x"
Jul 08 19:06:32.763714 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.763672    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4sk5h"]
Jul 08 19:06:32.767164 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:32.767076    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc2336f_9b1b_401d_89a6_dd190e97c7b5.slice/crio-cd9257cb84a685ab865e3de813cb955bed7d879d50f1de0efccf3f7196839906 WatchSource:0}: Error finding container cd9257cb84a685ab865e3de813cb955bed7d879d50f1de0efccf3f7196839906: Status 404 returned error can't find the container with id cd9257cb84a685ab865e3de813cb955bed7d879d50f1de0efccf3f7196839906
Jul 08 19:06:32.892563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.892529    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vrp9v"]
Jul 08 19:06:32.895386 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:32.895357    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0f6079_82a2_4e15_b3a4_786abf7ad66e.slice/crio-819b75051fa8397dc8c0235164cfa4048a1e3d31d799958ffa6364d33992f003 WatchSource:0}: Error finding container 819b75051fa8397dc8c0235164cfa4048a1e3d31d799958ffa6364d33992f003: Status 404 returned error can't find the container with id 819b75051fa8397dc8c0235164cfa4048a1e3d31d799958ffa6364d33992f003
Jul 08 19:06:32.904220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:32.904200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kxb2x"]
Jul 08 19:06:32.906509 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:32.906483    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ca6196_d827_45d8_985d_b0d8ffcd0f57.slice/crio-dd239ac845a6b6e2cbe0e64c96a0d21e432f4e9524f3ccc213f2811999363f71 WatchSource:0}: Error finding container dd239ac845a6b6e2cbe0e64c96a0d21e432f4e9524f3ccc213f2811999363f71: Status 404 returned error can't find the container with id dd239ac845a6b6e2cbe0e64c96a0d21e432f4e9524f3ccc213f2811999363f71
Jul 08 19:06:33.256343 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:33.256303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kxb2x" event={"ID":"69ca6196-d827-45d8-985d-b0d8ffcd0f57","Type":"ContainerStarted","Data":"dd239ac845a6b6e2cbe0e64c96a0d21e432f4e9524f3ccc213f2811999363f71"}
Jul 08 19:06:33.257446 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:33.257413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vrp9v" event={"ID":"7a0f6079-82a2-4e15-b3a4-786abf7ad66e","Type":"ContainerStarted","Data":"819b75051fa8397dc8c0235164cfa4048a1e3d31d799958ffa6364d33992f003"}
Jul 08 19:06:33.258544 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:33.258523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4sk5h" event={"ID":"bcc2336f-9b1b-401d-89a6-dd190e97c7b5","Type":"ContainerStarted","Data":"cd9257cb84a685ab865e3de813cb955bed7d879d50f1de0efccf3f7196839906"}
Jul 08 19:06:34.050871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:34.050824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:34.054093 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:34.054052    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b806e0a5-276a-4d25-93ec-e909b812ab48-metrics-tls\") pod \"dns-default-zfw8l\" (UID: \"b806e0a5-276a-4d25-93ec-e909b812ab48\") " pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:34.291684 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:34.291658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxdf4\""
Jul 08 19:06:34.299619 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:34.299566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:34.484723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:34.484696    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zfw8l"]
Jul 08 19:06:34.490627 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:34.490597    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb806e0a5_276a_4d25_93ec_e909b812ab48.slice/crio-6231ff2a1f9b2f59060d94bf24b754b81854b1f96b1fc253c23aeace19919cc7 WatchSource:0}: Error finding container 6231ff2a1f9b2f59060d94bf24b754b81854b1f96b1fc253c23aeace19919cc7: Status 404 returned error can't find the container with id 6231ff2a1f9b2f59060d94bf24b754b81854b1f96b1fc253c23aeace19919cc7
Jul 08 19:06:35.266198 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:35.266109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zfw8l" event={"ID":"b806e0a5-276a-4d25-93ec-e909b812ab48","Type":"ContainerStarted","Data":"6231ff2a1f9b2f59060d94bf24b754b81854b1f96b1fc253c23aeace19919cc7"}
Jul 08 19:06:35.270071 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:35.269283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kxb2x" event={"ID":"69ca6196-d827-45d8-985d-b0d8ffcd0f57","Type":"ContainerStarted","Data":"ee3a217d55dbd59a8d0abd1f27ec6b5fe3e32a88b1597fb75be7452eab9e94c2"}
Jul 08 19:06:35.270071 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:35.269339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kxb2x" event={"ID":"69ca6196-d827-45d8-985d-b0d8ffcd0f57","Type":"ContainerStarted","Data":"09222b304ae1db3376927956ad42644a35e9229c1731395ddc84c68a813f6118"}
Jul 08 19:06:36.771923 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:36.771873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:36.774826 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:36.774797    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63deeaca-7675-41e6-b345-191f40b70d31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4flzn\" (UID: \"63deeaca-7675-41e6-b345-191f40b70d31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:36.991873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:36.991837    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-96ml2\""
Jul 08 19:06:36.999307 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:36.999275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"
Jul 08 19:06:38.887646 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:38.887575    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kxb2x" podStartSLOduration=71.636859473 podStartE2EDuration="1m12.887551335s" totalImagesPullingTime="1.250691862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:35.291847856 +0000 UTC m=+68.956410412" watchObservedRunningTime="2026-07-08 19:06:38.887551335 +0000 UTC m=+72.552113888"
Jul 08 19:06:38.888133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:38.888038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4flzn"]
Jul 08 19:06:38.892243 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:38.892201    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63deeaca_7675_41e6_b345_191f40b70d31.slice/crio-eb14808d4f27d728d7eff400af80b672e5315a8d260f3de8158e44695fd91436 WatchSource:0}: Error finding container eb14808d4f27d728d7eff400af80b672e5315a8d260f3de8158e44695fd91436: Status 404 returned error can't find the container with id eb14808d4f27d728d7eff400af80b672e5315a8d260f3de8158e44695fd91436
Jul 08 19:06:39.191944 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.191904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:39.194421 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.194392    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96d21489-9846-4fac-aa04-91845fd9e46b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qw687\" (UID: \"96d21489-9846-4fac-aa04-91845fd9e46b\") " pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:39.283691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.283650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vrp9v" event={"ID":"7a0f6079-82a2-4e15-b3a4-786abf7ad66e","Type":"ContainerStarted","Data":"016dd19366820acf30f8e8e03eb71f79234815c159032671885ab1223765730e"}
Jul 08 19:06:39.283860 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.283758    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:06:39.285213 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.285183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4sk5h" event={"ID":"bcc2336f-9b1b-401d-89a6-dd190e97c7b5","Type":"ContainerStarted","Data":"a2d12ed45ac92c1659f5e715cb17e12bc07882e934a04f343b3ef92f147f441b"}
Jul 08 19:06:39.286843 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.286815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zfw8l" event={"ID":"b806e0a5-276a-4d25-93ec-e909b812ab48","Type":"ContainerStarted","Data":"5bc1bc3755943fa8afc6a2e62f712b2834053bee5beaae05ffda797a9295fba6"}
Jul 08 19:06:39.286985 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.286868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zfw8l" event={"ID":"b806e0a5-276a-4d25-93ec-e909b812ab48","Type":"ContainerStarted","Data":"41bad8ba741daea90421306b2adc4bc0c53419575f9cef51c7197e2c8d276fdd"}
Jul 08 19:06:39.286985 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.286900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:39.288029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.288009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn" event={"ID":"63deeaca-7675-41e6-b345-191f40b70d31","Type":"ContainerStarted","Data":"eb14808d4f27d728d7eff400af80b672e5315a8d260f3de8158e44695fd91436"}
Jul 08 19:06:39.299341 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.299296    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vrp9v" podStartSLOduration=67.452608117 podStartE2EDuration="1m13.299281662s" totalImagesPullingTime="5.846673545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:39.299097892 +0000 UTC m=+72.963660438" watchObservedRunningTime="2026-07-08 19:06:39.299281662 +0000 UTC m=+72.963844241"
Jul 08 19:06:39.317162 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.317113    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zfw8l" podStartSLOduration=33.06761939 podStartE2EDuration="37.317075618s" totalImagesPullingTime="4.249456228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:39.316244772 +0000 UTC m=+72.980807324" watchObservedRunningTime="2026-07-08 19:06:39.317075618 +0000 UTC m=+72.981638168"
Jul 08 19:06:39.332251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.332211    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4sk5h" podStartSLOduration=67.365516593 podStartE2EDuration="1m13.332198351s" totalImagesPullingTime="5.966681758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:39.331745975 +0000 UTC m=+72.996308527" watchObservedRunningTime="2026-07-08 19:06:39.332198351 +0000 UTC m=+72.996761172"
Jul 08 19:06:39.492147 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.492059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vczqd\""
Jul 08 19:06:39.500191 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.500161    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qw687"
Jul 08 19:06:39.519821 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.519790    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59878f9659-fdwn2"]
Jul 08 19:06:39.521857 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.521834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.525308 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.525286    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 19:06:39.525744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.525720    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 19:06:39.525865 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.525758    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 19:06:39.526746 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.526726    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 19:06:39.526969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.526942    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 19:06:39.526969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.526956    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 19:06:39.527139 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.526951    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 19:06:39.527198 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.527175    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-76gvt\""
Jul 08 19:06:39.538584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.538559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59878f9659-fdwn2"]
Jul 08 19:06:39.595033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.594999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-service-ca\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.595232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.595057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-oauth-serving-cert\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.595232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.595120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-oauth-config\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.595232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.595159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-config\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.595232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.595187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpqqq\" (UniqueName: \"kubernetes.io/projected/5294a0ca-9170-4746-992d-1bf51b1c72ac-kube-api-access-zpqqq\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.595414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.595265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-serving-cert\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.679831 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.679681    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qw687"]
Jul 08 19:06:39.683588 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:39.683553    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d21489_9846_4fac_aa04_91845fd9e46b.slice/crio-0996f95102d0ddb94317c854f67a5dcb5095be0577b5b8845531134ba9ef2be5 WatchSource:0}: Error finding container 0996f95102d0ddb94317c854f67a5dcb5095be0577b5b8845531134ba9ef2be5: Status 404 returned error can't find the container with id 0996f95102d0ddb94317c854f67a5dcb5095be0577b5b8845531134ba9ef2be5
Jul 08 19:06:39.696447 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.696411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-service-ca\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.696574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.696463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-oauth-serving-cert\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.696574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.696491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-oauth-config\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.696574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.696531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-config\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.696938 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.696895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqqq\" (UniqueName: \"kubernetes.io/projected/5294a0ca-9170-4746-992d-1bf51b1c72ac-kube-api-access-zpqqq\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.697039 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.696962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-serving-cert\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.697174 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.697150    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-service-ca\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.697263 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.697246    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-oauth-serving-cert\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.697263 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.697252    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-config\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.699333 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.699313    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-oauth-config\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.700060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.700039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-serving-cert\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.709990 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.709961    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqqq\" (UniqueName: \"kubernetes.io/projected/5294a0ca-9170-4746-992d-1bf51b1c72ac-kube-api-access-zpqqq\") pod \"console-59878f9659-fdwn2\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") " pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.832003 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.831921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:39.975247 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:39.975210    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59878f9659-fdwn2"]
Jul 08 19:06:40.145157 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:40.145124    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5294a0ca_9170_4746_992d_1bf51b1c72ac.slice/crio-a6c7cbfe8a24b520a84f84b8ec0d6fc81fdc801321133e38561525c8e2615105 WatchSource:0}: Error finding container a6c7cbfe8a24b520a84f84b8ec0d6fc81fdc801321133e38561525c8e2615105: Status 404 returned error can't find the container with id a6c7cbfe8a24b520a84f84b8ec0d6fc81fdc801321133e38561525c8e2615105
Jul 08 19:06:40.295823 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:40.295778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn" event={"ID":"63deeaca-7675-41e6-b345-191f40b70d31","Type":"ContainerStarted","Data":"d28391cb1b446891c712808eeb9ad2aa142452fabf2d5d7edf12d0688b3066dd"}
Jul 08 19:06:40.298571 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:40.298541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59878f9659-fdwn2" event={"ID":"5294a0ca-9170-4746-992d-1bf51b1c72ac","Type":"ContainerStarted","Data":"a6c7cbfe8a24b520a84f84b8ec0d6fc81fdc801321133e38561525c8e2615105"}
Jul 08 19:06:40.300180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:40.300058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qw687" event={"ID":"96d21489-9846-4fac-aa04-91845fd9e46b","Type":"ContainerStarted","Data":"0996f95102d0ddb94317c854f67a5dcb5095be0577b5b8845531134ba9ef2be5"}
Jul 08 19:06:45.325959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:45.325916    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59878f9659-fdwn2" event={"ID":"5294a0ca-9170-4746-992d-1bf51b1c72ac","Type":"ContainerStarted","Data":"4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf"}
Jul 08 19:06:45.327723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:45.327690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qw687" event={"ID":"96d21489-9846-4fac-aa04-91845fd9e46b","Type":"ContainerStarted","Data":"4d7a68bfe5e03f4a6df0ecbd10e4e2b4c5d28a51229b2a40a1b06ac1970ed827"}
Jul 08 19:06:45.347131 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:45.347053    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4flzn" podStartSLOduration=40.071403866 podStartE2EDuration="41.347030716s" totalImagesPullingTime="1.27562685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:40.318652778 +0000 UTC m=+73.983215333" watchObservedRunningTime="2026-07-08 19:06:45.347030716 +0000 UTC m=+79.011593268"
Jul 08 19:06:45.347371 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:45.347351    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59878f9659-fdwn2" podStartSLOduration=1.71617445 podStartE2EDuration="6.34734439s" totalImagesPullingTime="4.63116994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:45.345962604 +0000 UTC m=+79.010525172" watchObservedRunningTime="2026-07-08 19:06:45.34734439 +0000 UTC m=+79.011906997"
Jul 08 19:06:46.333995 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:46.333954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qw687" event={"ID":"96d21489-9846-4fac-aa04-91845fd9e46b","Type":"ContainerStarted","Data":"3ea495d40e125c1ef2504361f5907304a319747d2892ef5fe67b2698f515d5a1"}
Jul 08 19:06:48.343556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.343452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qw687" event={"ID":"96d21489-9846-4fac-aa04-91845fd9e46b","Type":"ContainerStarted","Data":"0e2531e4bd7884f2baba0e0374aa3e32d2bd7a73500e3866de848684bb5a9d28"}
Jul 08 19:06:48.364394 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.364343    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qw687" podStartSLOduration=33.282873808 podStartE2EDuration="41.364321904s" totalImagesPullingTime="8.081448096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:06:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:48.363832055 +0000 UTC m=+82.028394605" watchObservedRunningTime="2026-07-08 19:06:48.364321904 +0000 UTC m=+82.028884455"
Jul 08 19:06:48.870514 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.870476    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"]
Jul 08 19:06:48.874821 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.874787    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:48.877520 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.877494    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 19:06:48.877877 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.877804    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 08 19:06:48.878062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.878040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rh2gz\""
Jul 08 19:06:48.878194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.878104    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 08 19:06:48.878194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.878166    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 19:06:48.878323 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.877852    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 19:06:48.880281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.880245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"]
Jul 08 19:06:48.884069 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.884048    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:48.886287 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.886264    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 08 19:06:48.888293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.887873    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 08 19:06:48.888632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.887923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 08 19:06:48.888779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.887968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-khcst\""
Jul 08 19:06:48.888912 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.887988    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"]
Jul 08 19:06:48.888989 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.888969    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sjmck"]
Jul 08 19:06:48.893004 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.892976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.895687 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.895662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"]
Jul 08 19:06:48.896010 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.895989    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fq6bn\""
Jul 08 19:06:48.896317 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.896302    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 19:06:48.896487 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.896474    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 19:06:48.896643 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.896622    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 19:06:48.966978 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.966939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e1ff07f-b3bc-4464-b389-8c9117774155-metrics-client-ca\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.966978 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.966990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:48.967260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/70a7cdd0-4166-43c4-9fa7-bf7efde42935-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:48.967260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:48.967260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-textfile\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47xt2\" (UniqueName: \"kubernetes.io/projected/70a7cdd0-4166-43c4-9fa7-bf7efde42935-kube-api-access-47xt2\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70a7cdd0-4166-43c4-9fa7-bf7efde42935-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7dvt\" (UniqueName: \"kubernetes.io/projected/0e1ff07f-b3bc-4464-b389-8c9117774155-kube-api-access-w7dvt\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70a7cdd0-4166-43c4-9fa7-bf7efde42935-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-wtmp\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-sys\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:48.967559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxtqh\" (UniqueName: \"kubernetes.io/projected/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-api-access-mxtqh\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:48.967921 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-root\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967921 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-tls\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:48.967921 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:48.967635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.068251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-tls\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e1ff07f-b3bc-4464-b389-8c9117774155-metrics-client-ca\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/70a7cdd0-4166-43c4-9fa7-bf7efde42935-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:49.068379    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 19:06:49.068423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-textfile\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.068779 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:49.068453    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-tls podName:0e1ff07f-b3bc-4464-b389-8c9117774155 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:49.56843214 +0000 UTC m=+83.232994695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-tls") pod "node-exporter-sjmck" (UID: "0e1ff07f-b3bc-4464-b389-8c9117774155") : secret "node-exporter-tls" not found
Jul 08 19:06:49.068779 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:49.068501    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 08 19:06:49.068779 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:06:49.068542    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-tls podName:c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c nodeName:}" failed. No retries permitted until 2026-07-08 19:06:49.568529107 +0000 UTC m=+83.233091641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-w5kkn" (UID: "c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c") : secret "kube-state-metrics-tls" not found
Jul 08 19:06:49.068779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068704    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.068986 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.068862    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-textfile\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069038    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e1ff07f-b3bc-4464-b389-8c9117774155-metrics-client-ca\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069117    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.069154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069254 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069254 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47xt2\" (UniqueName: \"kubernetes.io/projected/70a7cdd0-4166-43c4-9fa7-bf7efde42935-kube-api-access-47xt2\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.069254 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70a7cdd0-4166-43c4-9fa7-bf7efde42935-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.069399 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.069399 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7dvt\" (UniqueName: \"kubernetes.io/projected/0e1ff07f-b3bc-4464-b389-8c9117774155-kube-api-access-w7dvt\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069399 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70a7cdd0-4166-43c4-9fa7-bf7efde42935-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.069399 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-wtmp\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069589 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-sys\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069589 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.069695 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069625    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-wtmp\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069695 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069631    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.069791 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069695    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-sys\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.070140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.069965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxtqh\" (UniqueName: \"kubernetes.io/projected/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-api-access-mxtqh\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.070140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.070006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-root\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.070140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.070110    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e1ff07f-b3bc-4464-b389-8c9117774155-root\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.070140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.070116    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.070351 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.070296    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70a7cdd0-4166-43c4-9fa7-bf7efde42935-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.071332 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.071310    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/70a7cdd0-4166-43c4-9fa7-bf7efde42935-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.071629 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.071609    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.072282 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.072261    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70a7cdd0-4166-43c4-9fa7-bf7efde42935-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.072425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.072407    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.081836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.081797    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7dvt\" (UniqueName: \"kubernetes.io/projected/0e1ff07f-b3bc-4464-b389-8c9117774155-kube-api-access-w7dvt\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.081964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.081854    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47xt2\" (UniqueName: \"kubernetes.io/projected/70a7cdd0-4166-43c4-9fa7-bf7efde42935-kube-api-access-47xt2\") pod \"openshift-state-metrics-54cf96dfcf-8pvfz\" (UID: \"70a7cdd0-4166-43c4-9fa7-bf7efde42935\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.083301 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.083274    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxtqh\" (UniqueName: \"kubernetes.io/projected/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-api-access-mxtqh\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.188486 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.188389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"
Jul 08 19:06:49.303383 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.303354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zfw8l"
Jul 08 19:06:49.329507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.328838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz"]
Jul 08 19:06:49.331989 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:49.331957    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a7cdd0_4166_43c4_9fa7_bf7efde42935.slice/crio-7d87761538a029d153464c8c9e86f05ea0d96e22b8fdccf94694231c86b8ef9c WatchSource:0}: Error finding container 7d87761538a029d153464c8c9e86f05ea0d96e22b8fdccf94694231c86b8ef9c: Status 404 returned error can't find the container with id 7d87761538a029d153464c8c9e86f05ea0d96e22b8fdccf94694231c86b8ef9c
Jul 08 19:06:49.350158 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.350127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz" event={"ID":"70a7cdd0-4166-43c4-9fa7-bf7efde42935","Type":"ContainerStarted","Data":"7d87761538a029d153464c8c9e86f05ea0d96e22b8fdccf94694231c86b8ef9c"}
Jul 08 19:06:49.574763 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.574715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-tls\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.574921 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.574764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.577287 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.577252    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e1ff07f-b3bc-4464-b389-8c9117774155-node-exporter-tls\") pod \"node-exporter-sjmck\" (UID: \"0e1ff07f-b3bc-4464-b389-8c9117774155\") " pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.577404 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.577299    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5kkn\" (UID: \"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.804759 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.804671    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"
Jul 08 19:06:49.812114 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.812068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sjmck"
Jul 08 19:06:49.823266 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:49.823232    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1ff07f_b3bc_4464_b389_8c9117774155.slice/crio-33b239b3807407bf064554527809c7a99430f1bc054c99d811d25a2fcb92161e WatchSource:0}: Error finding container 33b239b3807407bf064554527809c7a99430f1bc054c99d811d25a2fcb92161e: Status 404 returned error can't find the container with id 33b239b3807407bf064554527809c7a99430f1bc054c99d811d25a2fcb92161e
Jul 08 19:06:49.832727 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.832699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:49.832855 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.832817    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:49.838413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.838382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:49.953805 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:49.953770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn"]
Jul 08 19:06:49.958782 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:49.958749    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8fcddc1_6fa4_4c8e_bd0f_ac0ba0a27a8c.slice/crio-f00379102acbd8d016e204d289da8293b2c51557fb0bacc97635db7498fb0192 WatchSource:0}: Error finding container f00379102acbd8d016e204d289da8293b2c51557fb0bacc97635db7498fb0192: Status 404 returned error can't find the container with id f00379102acbd8d016e204d289da8293b2c51557fb0bacc97635db7498fb0192
Jul 08 19:06:50.355889 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:50.355849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjmck" event={"ID":"0e1ff07f-b3bc-4464-b389-8c9117774155","Type":"ContainerStarted","Data":"33b239b3807407bf064554527809c7a99430f1bc054c99d811d25a2fcb92161e"}
Jul 08 19:06:50.358244 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:50.358198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz" event={"ID":"70a7cdd0-4166-43c4-9fa7-bf7efde42935","Type":"ContainerStarted","Data":"60acd58f8d87a191d6deebda752096760e90265a9b23c93b033123f2b1f44bb4"}
Jul 08 19:06:50.358406 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:50.358280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz" event={"ID":"70a7cdd0-4166-43c4-9fa7-bf7efde42935","Type":"ContainerStarted","Data":"97e7444c2bea3c63fcf85c1888714e33ccabaa1801d2e53a5c8bcbb9ed3ba9c9"}
Jul 08 19:06:50.359648 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:50.359619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn" event={"ID":"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c","Type":"ContainerStarted","Data":"f00379102acbd8d016e204d289da8293b2c51557fb0bacc97635db7498fb0192"}
Jul 08 19:06:50.364495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:50.364470    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:06:51.365165 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:51.365130    2563 generic.go:356] "Generic (PLEG): container finished" podID="0e1ff07f-b3bc-4464-b389-8c9117774155" containerID="feffd171ecb70b4243b3ab79c3177ef1283867c3217007189d2b09683f54b01e" exitCode=0
Jul 08 19:06:51.365615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:51.365222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjmck" event={"ID":"0e1ff07f-b3bc-4464-b389-8c9117774155","Type":"ContainerDied","Data":"feffd171ecb70b4243b3ab79c3177ef1283867c3217007189d2b09683f54b01e"}
Jul 08 19:06:51.368948 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:51.368916    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz" event={"ID":"70a7cdd0-4166-43c4-9fa7-bf7efde42935","Type":"ContainerStarted","Data":"6f0caf5e306100e256a8f2ec0ecf68a9a70bbcce0bdf9509b95ffc4b84bed7e4"}
Jul 08 19:06:51.405852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:51.405794    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8pvfz" podStartSLOduration=2.386445768 podStartE2EDuration="3.405773086s" totalImagesPullingTime="1.019327318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:51.404959813 +0000 UTC m=+85.069522365" watchObservedRunningTime="2026-07-08 19:06:51.405773086 +0000 UTC m=+85.070335638"
Jul 08 19:06:51.941651 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:51.941616    2563 patch_prober.go:28] interesting pod/image-registry-8668666774-m8zdk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 19:06:51.941836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:51.941668    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8668666774-m8zdk" podUID="16289e5d-6029-465a-b20d-7138d7ab4206" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 19:06:52.376008 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.375957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjmck" event={"ID":"0e1ff07f-b3bc-4464-b389-8c9117774155","Type":"ContainerStarted","Data":"f7ec43d0b46c1f68008d1c0e73f390f35ad7890b34a14b8a88a352cd4e5f4858"}
Jul 08 19:06:52.376458 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.376030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjmck" event={"ID":"0e1ff07f-b3bc-4464-b389-8c9117774155","Type":"ContainerStarted","Data":"fc7445e9dbcdea811a22f9f7f6044d3369277845f32fe099dbef612e90b51f95"}
Jul 08 19:06:52.378688 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.378659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn" event={"ID":"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c","Type":"ContainerStarted","Data":"aa2f67179b07adc40b24656a9b193bf86d3a704ec4f8dad66be7866add48064a"}
Jul 08 19:06:52.378902 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.378728    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn" event={"ID":"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c","Type":"ContainerStarted","Data":"d0417dd0454284efe0f5d10c5525111551c969f35eecb68b31cdda5c5e40a58f"}
Jul 08 19:06:52.383134 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.379249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn" event={"ID":"c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c","Type":"ContainerStarted","Data":"a0194243b676eed17d78ca23d34cee43834ed0a22950ef73eba7eaf2e2a47674"}
Jul 08 19:06:52.427116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.427047    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sjmck" podStartSLOduration=3.145709603 podStartE2EDuration="4.427032234s" totalImagesPullingTime="1.198340383s" totalInitContainerRuntime="82.982248ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:52.426229342 +0000 UTC m=+86.090791894" watchObservedRunningTime="2026-07-08 19:06:52.427032234 +0000 UTC m=+86.091594784"
Jul 08 19:06:52.475097 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:52.475038    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5kkn" podStartSLOduration=2.559345397 podStartE2EDuration="4.475019897s" totalImagesPullingTime="1.9156745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:52.473748434 +0000 UTC m=+86.138310996" watchObservedRunningTime="2026-07-08 19:06:52.475019897 +0000 UTC m=+86.139582460"
Jul 08 19:06:53.263250 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.263220    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8668666774-m8zdk"
Jul 08 19:06:53.674983 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.674942    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"]
Jul 08 19:06:53.678268 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.678248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.681281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.681254    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 08 19:06:53.681281 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.681272    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6grnn\""
Jul 08 19:06:53.690427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.690402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"]
Jul 08 19:06:53.811333 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.811293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s628\" (UniqueName: \"kubernetes.io/projected/d07ea37b-58da-4500-a70b-6b8eb46fa373-kube-api-access-5s628\") pod \"monitoring-plugin-d5dcff7cf-skl47\" (UID: \"d07ea37b-58da-4500-a70b-6b8eb46fa373\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.811333 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.811350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d07ea37b-58da-4500-a70b-6b8eb46fa373-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-skl47\" (UID: \"d07ea37b-58da-4500-a70b-6b8eb46fa373\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.912023 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.911982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s628\" (UniqueName: \"kubernetes.io/projected/d07ea37b-58da-4500-a70b-6b8eb46fa373-kube-api-access-5s628\") pod \"monitoring-plugin-d5dcff7cf-skl47\" (UID: \"d07ea37b-58da-4500-a70b-6b8eb46fa373\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.912023 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.912028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d07ea37b-58da-4500-a70b-6b8eb46fa373-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-skl47\" (UID: \"d07ea37b-58da-4500-a70b-6b8eb46fa373\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.914654 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.914633    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d07ea37b-58da-4500-a70b-6b8eb46fa373-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-skl47\" (UID: \"d07ea37b-58da-4500-a70b-6b8eb46fa373\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.922348 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.922316    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s628\" (UniqueName: \"kubernetes.io/projected/d07ea37b-58da-4500-a70b-6b8eb46fa373-kube-api-access-5s628\") pod \"monitoring-plugin-d5dcff7cf-skl47\" (UID: \"d07ea37b-58da-4500-a70b-6b8eb46fa373\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:53.990755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:53.990655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:54.123341 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:54.123313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"]
Jul 08 19:06:54.125833 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:54.125806    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07ea37b_58da_4500_a70b_6b8eb46fa373.slice/crio-974748ac82265b9da662404ffc6cd87db3687496b7c6d9a38a1a6fd9bc315d34 WatchSource:0}: Error finding container 974748ac82265b9da662404ffc6cd87db3687496b7c6d9a38a1a6fd9bc315d34: Status 404 returned error can't find the container with id 974748ac82265b9da662404ffc6cd87db3687496b7c6d9a38a1a6fd9bc315d34
Jul 08 19:06:54.387210 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:54.387179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47" event={"ID":"d07ea37b-58da-4500-a70b-6b8eb46fa373","Type":"ContainerStarted","Data":"974748ac82265b9da662404ffc6cd87db3687496b7c6d9a38a1a6fd9bc315d34"}
Jul 08 19:06:55.261972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.261929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:06:55.266296 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.266263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.277715 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.276432    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 19:06:55.277715 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.277248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 19:06:55.277715 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.277485    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 19:06:55.280551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.280518    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 19:06:55.305283 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305246    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 19:06:55.305477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305376    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 19:06:55.305477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305465    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2oamnnchfhlov\""
Jul 08 19:06:55.305609 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305510    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 19:06:55.305609 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305563    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hkfwq\""
Jul 08 19:06:55.305728 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305711    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 19:06:55.305786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305738    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 19:06:55.305786 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305767    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 19:06:55.306022 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.305998    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 19:06:55.306228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.306032    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 19:06:55.311981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.311956    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 19:06:55.313726 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.313702    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:06:55.324110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gbrr\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-kube-api-access-9gbrr\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.324880 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.324876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.329408 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.329374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 19:06:55.330732 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.330696    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 19:06:55.425346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gbrr\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-kube-api-access-9gbrr\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.425969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.425914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.426516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.426492    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.426584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.426561    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.426690 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.426664    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.427178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.426954    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.427447 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.427297    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.428098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.428029    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.428805 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.428784    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.429531 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.429512    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.430121 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.430071    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.430661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.430640    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.431285 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.430875    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.431285 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.431244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.431439 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.431371    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.431439 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.431401    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.432182 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.432160    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.432362 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.432345    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.432480 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.432461    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.432843 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.432822    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.433041 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.433022    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.437138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.437108    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gbrr\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-kube-api-access-9gbrr\") pod \"prometheus-k8s-0\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.589473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.589381    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:06:55.750756 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:55.750718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:06:56.005483 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:06:56.005453    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode340706a_5e08_4683_b2da_5e4dfd52ce4a.slice/crio-0f3ef04be52e26aa6634a6728eb443201d42697a020e2762c7eca2d45b1625a5 WatchSource:0}: Error finding container 0f3ef04be52e26aa6634a6728eb443201d42697a020e2762c7eca2d45b1625a5: Status 404 returned error can't find the container with id 0f3ef04be52e26aa6634a6728eb443201d42697a020e2762c7eca2d45b1625a5
Jul 08 19:06:56.395847 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:56.395811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47" event={"ID":"d07ea37b-58da-4500-a70b-6b8eb46fa373","Type":"ContainerStarted","Data":"e718d6e0b95d5557570b05d4241cf8a0a93754b7a03bb915ab99f7e9cfd2b48a"}
Jul 08 19:06:56.396325 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:56.396019    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:56.397219 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:56.397198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"0f3ef04be52e26aa6634a6728eb443201d42697a020e2762c7eca2d45b1625a5"}
Jul 08 19:06:56.400963 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:56.400947    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47"
Jul 08 19:06:56.413298 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:56.413241    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-skl47" podStartSLOduration=1.484951661 podStartE2EDuration="3.413224153s" totalImagesPullingTime="1.928272492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:56.412557711 +0000 UTC m=+90.077120265" watchObservedRunningTime="2026-07-08 19:06:56.413224153 +0000 UTC m=+90.077786709"
Jul 08 19:06:58.406041 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:58.406006    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf" exitCode=0
Jul 08 19:06:58.406447 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:06:58.406112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"}
Jul 08 19:07:03.429033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:03.429000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"}
Jul 08 19:07:03.429437 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:03.429066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"}
Jul 08 19:07:05.440885 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:05.440845    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"}
Jul 08 19:07:05.441284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:05.440928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"}
Jul 08 19:07:05.441284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:05.440969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"}
Jul 08 19:07:05.441284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:05.441006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerStarted","Data":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"}
Jul 08 19:07:05.475943 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:05.475884    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.625873543 podStartE2EDuration="10.475865248s" totalImagesPullingTime="7.832118613s" totalInitContainerRuntime="17.873092ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:05.473494202 +0000 UTC m=+99.138056752" watchObservedRunningTime="2026-07-08 19:07:05.475865248 +0000 UTC m=+99.140427799"
Jul 08 19:07:05.590272 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:05.590235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:07:09.978574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:09.978536    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59878f9659-fdwn2"]
Jul 08 19:07:10.303486 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:10.303404    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vrp9v"
Jul 08 19:07:35.000549 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.000502    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-59878f9659-fdwn2" podUID="5294a0ca-9170-4746-992d-1bf51b1c72ac" containerName="console" containerID="cri-o://4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf" gracePeriod=15
Jul 08 19:07:35.239429 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.239401    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:07:35.271649 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.271554    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-oauth-config\") pod \"5294a0ca-9170-4746-992d-1bf51b1c72ac\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") "
Jul 08 19:07:35.271649 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.271597    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-serving-cert\") pod \"5294a0ca-9170-4746-992d-1bf51b1c72ac\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") "
Jul 08 19:07:35.271649 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.271629    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-oauth-serving-cert\") pod \"5294a0ca-9170-4746-992d-1bf51b1c72ac\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") "
Jul 08 19:07:35.271922 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.271757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-config\") pod \"5294a0ca-9170-4746-992d-1bf51b1c72ac\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") "
Jul 08 19:07:35.271922 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.271842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpqqq\" (UniqueName: \"kubernetes.io/projected/5294a0ca-9170-4746-992d-1bf51b1c72ac-kube-api-access-zpqqq\") pod \"5294a0ca-9170-4746-992d-1bf51b1c72ac\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") "
Jul 08 19:07:35.271922 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.271899    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-service-ca\") pod \"5294a0ca-9170-4746-992d-1bf51b1c72ac\" (UID: \"5294a0ca-9170-4746-992d-1bf51b1c72ac\") "
Jul 08 19:07:35.272061 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.272019    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5294a0ca-9170-4746-992d-1bf51b1c72ac" (UID: "5294a0ca-9170-4746-992d-1bf51b1c72ac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:07:35.272179 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.272144    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-config" (OuterVolumeSpecName: "console-config") pod "5294a0ca-9170-4746-992d-1bf51b1c72ac" (UID: "5294a0ca-9170-4746-992d-1bf51b1c72ac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:07:35.272574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.272214    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-oauth-serving-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:07:35.272574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.272372    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-service-ca" (OuterVolumeSpecName: "service-ca") pod "5294a0ca-9170-4746-992d-1bf51b1c72ac" (UID: "5294a0ca-9170-4746-992d-1bf51b1c72ac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:07:35.274031 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.274005    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5294a0ca-9170-4746-992d-1bf51b1c72ac" (UID: "5294a0ca-9170-4746-992d-1bf51b1c72ac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:07:35.274171 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.274148    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5294a0ca-9170-4746-992d-1bf51b1c72ac" (UID: "5294a0ca-9170-4746-992d-1bf51b1c72ac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:07:35.274234 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.274207    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5294a0ca-9170-4746-992d-1bf51b1c72ac-kube-api-access-zpqqq" (OuterVolumeSpecName: "kube-api-access-zpqqq") pod "5294a0ca-9170-4746-992d-1bf51b1c72ac" (UID: "5294a0ca-9170-4746-992d-1bf51b1c72ac"). InnerVolumeSpecName "kube-api-access-zpqqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:07:35.373593 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.373553    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:07:35.373755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.373609    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-oauth-config\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:07:35.373755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.373620    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-serving-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:07:35.373755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.373629    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5294a0ca-9170-4746-992d-1bf51b1c72ac-console-config\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:07:35.373755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.373637    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpqqq\" (UniqueName: \"kubernetes.io/projected/5294a0ca-9170-4746-992d-1bf51b1c72ac-kube-api-access-zpqqq\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:07:35.554547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.554454    2563 generic.go:356] "Generic (PLEG): container finished" podID="5294a0ca-9170-4746-992d-1bf51b1c72ac" containerID="4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf" exitCode=0
Jul 08 19:07:35.554547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.554501    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59878f9659-fdwn2"
Jul 08 19:07:35.554547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.554526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59878f9659-fdwn2" event={"ID":"5294a0ca-9170-4746-992d-1bf51b1c72ac","Type":"ContainerDied","Data":"4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf"}
Jul 08 19:07:35.554782 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.554576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59878f9659-fdwn2" event={"ID":"5294a0ca-9170-4746-992d-1bf51b1c72ac","Type":"ContainerDied","Data":"a6c7cbfe8a24b520a84f84b8ec0d6fc81fdc801321133e38561525c8e2615105"}
Jul 08 19:07:35.554782 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.554618    2563 scope.go:117] "RemoveContainer" containerID="4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf"
Jul 08 19:07:35.563932 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.563913    2563 scope.go:117] "RemoveContainer" containerID="4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf"
Jul 08 19:07:35.564225 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:07:35.564206    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf\": container with ID starting with 4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf not found: ID does not exist" containerID="4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf"
Jul 08 19:07:35.564292 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.564236    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf"} err="failed to get container status \"4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf\": rpc error: code = NotFound desc = could not find container \"4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf\": container with ID starting with 4da2f6e8a84860fa1a228e91b25b4043222c0e1489ccf671197e47fd22e7deaf not found: ID does not exist"
Jul 08 19:07:35.578313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.578280    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59878f9659-fdwn2"]
Jul 08 19:07:35.582197 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:35.582172    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59878f9659-fdwn2"]
Jul 08 19:07:36.929097 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:36.929057    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5294a0ca-9170-4746-992d-1bf51b1c72ac" path="/var/lib/kubelet/pods/5294a0ca-9170-4746-992d-1bf51b1c72ac/volumes"
Jul 08 19:07:55.590493 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:55.590450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:07:55.612779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:55.612748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:07:55.647892 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:07:55.647864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:13.610916 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.610882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:08:13.611448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.611200    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="prometheus" containerID="cri-o://1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1" gracePeriod=600
Jul 08 19:08:13.611448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.611245    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy" containerID="cri-o://dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f" gracePeriod=600
Jul 08 19:08:13.611448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.611274    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-web" containerID="cri-o://dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2" gracePeriod=600
Jul 08 19:08:13.611448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.611277    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="config-reloader" containerID="cri-o://f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1" gracePeriod=600
Jul 08 19:08:13.611448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.611265    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="thanos-sidecar" containerID="cri-o://5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d" gracePeriod=600
Jul 08 19:08:13.611448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.611325    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590" gracePeriod=600
Jul 08 19:08:13.849311 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:13.849277    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.001744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gbrr\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-kube-api-access-9gbrr\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.001744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001693    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-serving-certs-ca-bundle\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.001744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-metrics-client-ca\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.001744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-1\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001754    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001773    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-trusted-ca-bundle\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001798    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-tls-assets\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001833    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001858    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config-out\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-2\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001918    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-tls\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.001967    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-web-config\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002000    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002039    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-db\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002073 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002074    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-0\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-thanos-prometheus-http-client-file\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002143    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002176    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-kubelet-serving-ca-bundle\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002200    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002225    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-kube-rbac-proxy\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002234    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002253    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-metrics-client-certs\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002282    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-grpc-tls\") pod \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\" (UID: \"e340706a-5e08-4683-b2da-5e4dfd52ce4a\") "
Jul 08 19:08:14.002644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002515    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.003128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002657    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.003128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002698    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.003128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002720    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.003128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.002737    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.003804 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.003772    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.005125 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.004880    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-kube-api-access-9gbrr" (OuterVolumeSpecName: "kube-api-access-9gbrr") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "kube-api-access-9gbrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:08:14.005125 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.004983    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.005125 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.005013    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config" (OuterVolumeSpecName: "config") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.005125 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.005028    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.005425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.005154    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:08:14.005425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.005283    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.005425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.005343    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.005593 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.005512    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:08:14.006907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.006881    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:08:14.007032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.007014    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config-out" (OuterVolumeSpecName: "config-out") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:08:14.007640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.007624    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.008046 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.008030    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.008115 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.008050    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.017858 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.017830    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-web-config" (OuterVolumeSpecName: "web-config") pod "e340706a-5e08-4683-b2da-5e4dfd52ce4a" (UID: "e340706a-5e08-4683-b2da-5e4dfd52ce4a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:08:14.103637 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103597    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-kube-rbac-proxy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103637 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103629    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-metrics-client-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103637 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103644    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-grpc-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103656    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gbrr\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-kube-api-access-9gbrr\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103670    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-metrics-client-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103683    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103696    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e340706a-5e08-4683-b2da-5e4dfd52ce4a-tls-assets\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103709    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103722    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-config-out\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103734    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103745    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103757    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-web-config\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103769    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103781    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-db\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103795    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103808    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e340706a-5e08-4683-b2da-5e4dfd52ce4a-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.103838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.103822    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e340706a-5e08-4683-b2da-5e4dfd52ce4a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:08:14.703007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.702968    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590" exitCode=0
Jul 08 19:08:14.703007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.702998    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f" exitCode=0
Jul 08 19:08:14.703007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703008    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2" exitCode=0
Jul 08 19:08:14.703007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703016    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d" exitCode=0
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703025    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1" exitCode=0
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703032    2563 generic.go:356] "Generic (PLEG): container finished" podID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1" exitCode=0
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"}
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703066    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"}
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"}
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"}
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703197    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"}
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"}
Jul 08 19:08:14.703526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.703321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e340706a-5e08-4683-b2da-5e4dfd52ce4a","Type":"ContainerDied","Data":"0f3ef04be52e26aa6634a6728eb443201d42697a020e2762c7eca2d45b1625a5"}
Jul 08 19:08:14.711458 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.711431    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.718860 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.718840    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.726177 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.726151    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.729676 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.729650    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:08:14.735585 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.735561    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.736070 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.736041    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:08:14.745550 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.745500    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.753434 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.753410    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.760914 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.760893    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.761312 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.761288    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.761401 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.761322    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"} err="failed to get container status \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist"
Jul 08 19:08:14.761401 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.761344    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.761644 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.761629    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.761683 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.761653    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"} err="failed to get container status \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist"
Jul 08 19:08:14.761683 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.761668    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.761948 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.761929    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.762002 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.761953    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"} err="failed to get container status \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist"
Jul 08 19:08:14.762002 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.761985    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.762245 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.762226    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.762291 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762249    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"} err="failed to get container status \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist"
Jul 08 19:08:14.762291 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762263    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.762489 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.762474    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.762529 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762491    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"} err="failed to get container status \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist"
Jul 08 19:08:14.762529 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762506    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.762727 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.762705    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.762767 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762732    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"} err="failed to get container status \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist"
Jul 08 19:08:14.762767 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762747    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.762949 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:08:14.762933    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.762989 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762952    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"} err="failed to get container status \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist"
Jul 08 19:08:14.762989 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.762965    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.763217 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763199    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"} err="failed to get container status \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist"
Jul 08 19:08:14.763275 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763218    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.763445 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763418    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"} err="failed to get container status \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist"
Jul 08 19:08:14.763487 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763446    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.763670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763650    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"} err="failed to get container status \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist"
Jul 08 19:08:14.763715 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763671    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.763882 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763866    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"} err="failed to get container status \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist"
Jul 08 19:08:14.763928 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.763882    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.764077 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764062    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"} err="failed to get container status \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist"
Jul 08 19:08:14.764143 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764076    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.764319 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764298    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"} err="failed to get container status \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist"
Jul 08 19:08:14.764387 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764321    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.764548 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764531    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"} err="failed to get container status \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist"
Jul 08 19:08:14.764597 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764549    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.764750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764733    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"} err="failed to get container status \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist"
Jul 08 19:08:14.764792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764751    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.764931 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764913    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"} err="failed to get container status \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist"
Jul 08 19:08:14.764931 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.764930    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.765133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765117    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"} err="failed to get container status \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist"
Jul 08 19:08:14.765133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765133    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.765298 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765281    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"} err="failed to get container status \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist"
Jul 08 19:08:14.765349 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765298    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.765515 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765500    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"} err="failed to get container status \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist"
Jul 08 19:08:14.765551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765515    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.765721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765705    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"} err="failed to get container status \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist"
Jul 08 19:08:14.765754 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765723    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.765921 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765891    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"} err="failed to get container status \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist"
Jul 08 19:08:14.765964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.765921    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.766181 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766100    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"} err="failed to get container status \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist"
Jul 08 19:08:14.766181 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766116    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.766293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766276    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"} err="failed to get container status \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist"
Jul 08 19:08:14.766293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766288    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.766446 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766432    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"} err="failed to get container status \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist"
Jul 08 19:08:14.766491 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766447    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.766632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766614    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"} err="failed to get container status \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist"
Jul 08 19:08:14.766632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766629    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.766819 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766801    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"} err="failed to get container status \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist"
Jul 08 19:08:14.766885 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.766825    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.767046 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767031    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"} err="failed to get container status \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist"
Jul 08 19:08:14.767117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767046    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.767271 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767252    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"} err="failed to get container status \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist"
Jul 08 19:08:14.767271 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767270    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.767446 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767428    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"} err="failed to get container status \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist"
Jul 08 19:08:14.767446 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767445    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.767631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767613    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"} err="failed to get container status \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist"
Jul 08 19:08:14.767673 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767631    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.767810 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767795    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"} err="failed to get container status \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist"
Jul 08 19:08:14.767852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767810    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.767999 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767975    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"} err="failed to get container status \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist"
Jul 08 19:08:14.767999 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.767998    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.768294 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.768274    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"} err="failed to get container status \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist"
Jul 08 19:08:14.768294 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.768294    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.768787 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.768759    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"} err="failed to get container status \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist"
Jul 08 19:08:14.768787 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.768787    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.769133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.769107    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"} err="failed to get container status \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist"
Jul 08 19:08:14.769133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.769133    2563 scope.go:117] "RemoveContainer" containerID="1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"
Jul 08 19:08:14.769412 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.769382    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590"} err="failed to get container status \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": rpc error: code = NotFound desc = could not find container \"1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590\": container with ID starting with 1fc0471cf0c10549ff8984dd258d67011c79a58252cdac6a773bcd2a166a5590 not found: ID does not exist"
Jul 08 19:08:14.769412 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.769410    2563 scope.go:117] "RemoveContainer" containerID="dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"
Jul 08 19:08:14.769787 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.769757    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f"} err="failed to get container status \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": rpc error: code = NotFound desc = could not find container \"dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f\": container with ID starting with dbd93dba041a38f4a9820066f6b51ee78317c091225b93a5fed6694a3d0e133f not found: ID does not exist"
Jul 08 19:08:14.769787 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.769783    2563 scope.go:117] "RemoveContainer" containerID="dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"
Jul 08 19:08:14.770111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770069    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2"} err="failed to get container status \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": rpc error: code = NotFound desc = could not find container \"dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2\": container with ID starting with dc2a05404e1ee933e1c61df382037f9fc43f3262231cb4b44081a716b8d3a8d2 not found: ID does not exist"
Jul 08 19:08:14.770111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770111    2563 scope.go:117] "RemoveContainer" containerID="5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"
Jul 08 19:08:14.770347 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770329    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d"} err="failed to get container status \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": rpc error: code = NotFound desc = could not find container \"5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d\": container with ID starting with 5540c1262df00c72da054aafedcbfa0047586c830df479b1e0e71992c6f8c79d not found: ID does not exist"
Jul 08 19:08:14.770389 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770349    2563 scope.go:117] "RemoveContainer" containerID="f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"
Jul 08 19:08:14.770455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770435    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:08:14.770591 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770574    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1"} err="failed to get container status \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": rpc error: code = NotFound desc = could not find container \"f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1\": container with ID starting with f4de354ff5a6c76ae6f375f9898c2233b94239ca4a94082cf247c3a60afebcb1 not found: ID does not exist"
Jul 08 19:08:14.770649 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770592    2563 scope.go:117] "RemoveContainer" containerID="1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"
Jul 08 19:08:14.770808 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770794    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5294a0ca-9170-4746-992d-1bf51b1c72ac" containerName="console"
Jul 08 19:08:14.770845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770804    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1"} err="failed to get container status \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": rpc error: code = NotFound desc = could not find container \"1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1\": container with ID starting with 1d721a64b99eae375b42beac7e749b378213ecbb619d40af787f13507a3776d1 not found: ID does not exist"
Jul 08 19:08:14.770845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770821    2563 scope.go:117] "RemoveContainer" containerID="344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770811    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5294a0ca-9170-4746-992d-1bf51b1c72ac" containerName="console"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770864    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-thanos"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770872    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-thanos"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770881    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="init-config-reloader"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770887    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="init-config-reloader"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770897    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-web"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770902    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-web"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770910    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="config-reloader"
Jul 08 19:08:14.770919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="config-reloader"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770923    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="prometheus"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770931    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="prometheus"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770939    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770945    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770959    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="thanos-sidecar"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.770966    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="thanos-sidecar"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771032    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771041    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="config-reloader"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771033    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf"} err="failed to get container status \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": rpc error: code = NotFound desc = could not find container \"344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf\": container with ID starting with 344f8f18a965db8c55fb38cce311f59013c473173089839b1c8556f52e921aaf not found: ID does not exist"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771047    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-web"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771053    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="prometheus"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771058    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="thanos-sidecar"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771062    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" containerName="kube-rbac-proxy-thanos"
Jul 08 19:08:14.771222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.771067    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5294a0ca-9170-4746-992d-1bf51b1c72ac" containerName="console"
Jul 08 19:08:14.775863 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.775836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.780004 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.779980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 19:08:14.780246 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.780231    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 19:08:14.780337 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.780242    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 19:08:14.780637 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.780622    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 19:08:14.780720 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.780706    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 19:08:14.780966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.780952    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 19:08:14.781238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.781219    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2oamnnchfhlov\""
Jul 08 19:08:14.781415 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.781262    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 19:08:14.781415 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.781284    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 19:08:14.781415 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.781363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 19:08:14.781590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.781477    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 19:08:14.781695 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.781659    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hkfwq\""
Jul 08 19:08:14.782298 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.782279    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 19:08:14.783453 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.783431    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 19:08:14.783573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.783460    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 19:08:14.785268 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.785249    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 19:08:14.789702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.789680    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 19:08:14.810109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.794697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:08:14.909154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e9952d8-fe52-454e-ab54-96286003e799-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-config\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909462 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909492    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz5hn\" (UniqueName: \"kubernetes.io/projected/8e9952d8-fe52-454e-ab54-96286003e799-kube-api-access-wz5hn\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.909783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.909713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e9952d8-fe52-454e-ab54-96286003e799-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:14.928957 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:14.928911    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e340706a-5e08-4683-b2da-5e4dfd52ce4a" path="/var/lib/kubelet/pods/e340706a-5e08-4683-b2da-5e4dfd52ce4a/volumes"
Jul 08 19:08:15.010699 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.010699 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.010873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.010873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.010966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.010966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz5hn\" (UniqueName: \"kubernetes.io/projected/8e9952d8-fe52-454e-ab54-96286003e799-kube-api-access-wz5hn\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.010966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.010976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e9952d8-fe52-454e-ab54-96286003e799-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e9952d8-fe52-454e-ab54-96286003e799-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011269    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011593 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.011593 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.012691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.011733    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.012691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.012051    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.012691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.012415    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.012691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.012677    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014246    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014305    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014377    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014481    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e9952d8-fe52-454e-ab54-96286003e799-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014496    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-config\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.014760    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.015265    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e9952d8-fe52-454e-ab54-96286003e799-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.015464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.015390    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.016077 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.015624    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.016077 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.015639    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.016868 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.016845    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.017273 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.017251    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.017615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.017590    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e9952d8-fe52-454e-ab54-96286003e799-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.017731 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.017716    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e9952d8-fe52-454e-ab54-96286003e799-config\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.020906 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.020888    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz5hn\" (UniqueName: \"kubernetes.io/projected/8e9952d8-fe52-454e-ab54-96286003e799-kube-api-access-wz5hn\") pod \"prometheus-k8s-0\" (UID: \"8e9952d8-fe52-454e-ab54-96286003e799\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.085955 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.085916    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:08:15.244411 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.244374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 19:08:15.247574 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:08:15.247538    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9952d8_fe52_454e_ab54_96286003e799.slice/crio-08d9819968326ac8ee63f9931c74c6db7cdba1d648d15f1720b51b73d3181ebb WatchSource:0}: Error finding container 08d9819968326ac8ee63f9931c74c6db7cdba1d648d15f1720b51b73d3181ebb: Status 404 returned error can't find the container with id 08d9819968326ac8ee63f9931c74c6db7cdba1d648d15f1720b51b73d3181ebb
Jul 08 19:08:15.709706 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.709675    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e9952d8-fe52-454e-ab54-96286003e799" containerID="2ad0aa35c00421588e3ad18742c626470396c8d7177efb541a5627350c614e8c" exitCode=0
Jul 08 19:08:15.710124 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.709773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerDied","Data":"2ad0aa35c00421588e3ad18742c626470396c8d7177efb541a5627350c614e8c"}
Jul 08 19:08:15.710124 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:15.709843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"08d9819968326ac8ee63f9931c74c6db7cdba1d648d15f1720b51b73d3181ebb"}
Jul 08 19:08:16.717752 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.717654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"930b6cb75ec0b27493900e98a663c57cd0df1f9e72019af172cdd9362c5a5525"}
Jul 08 19:08:16.717752 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.717738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"ea9d1d46c39e5c77a4589708a4d04637e6b8065c844b4b1d9eb05f509e55b8fa"}
Jul 08 19:08:16.718302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.717778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"c5b1b4111e53501851bc966a28f00b8eb0ef94f8b40667788f017af16d9c7cd0"}
Jul 08 19:08:16.718302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.717814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"53f61444ab41cb6184e0afc05476193a08f2aae38941cd946693e08a30ad8fef"}
Jul 08 19:08:16.718302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.717854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"0e99f6ad257fbc439b853f870799a60168e26ed7fc8c85671ad13151004d7806"}
Jul 08 19:08:16.718302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.717894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e9952d8-fe52-454e-ab54-96286003e799","Type":"ContainerStarted","Data":"7519b7b01de37da7d2f6f83c1ad65173c6a0935bb26fbec546127925966b2173"}
Jul 08 19:08:16.765366 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:16.765307    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.745427405 podStartE2EDuration="2.765288377s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.860972ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:08:16.762986125 +0000 UTC m=+170.427548677" watchObservedRunningTime="2026-07-08 19:08:16.765288377 +0000 UTC m=+170.429850927"
Jul 08 19:08:20.086313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:08:20.086270    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:09:15.086638 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:09:15.086587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:09:15.102978 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:09:15.102951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:09:15.942616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:09:15.942579    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 19:10:26.743498 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:10:26.743468    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 19:11:13.225475 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.225437    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mnd5j"]
Jul 08 19:11:13.228619 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.228598    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.233207 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.233181    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 08 19:11:13.233207 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.233200    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-878l5\""
Jul 08 19:11:13.233849 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.233835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 08 19:11:13.257253 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.257218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mnd5j"]
Jul 08 19:11:13.257702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.257682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr74j\" (UniqueName: \"kubernetes.io/projected/657bb57a-8488-4e39-a740-d090633aefe9-kube-api-access-jr74j\") pod \"cert-manager-55b4d954d5-mnd5j\" (UID: \"657bb57a-8488-4e39-a740-d090633aefe9\") " pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.257806 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.257713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/657bb57a-8488-4e39-a740-d090633aefe9-bound-sa-token\") pod \"cert-manager-55b4d954d5-mnd5j\" (UID: \"657bb57a-8488-4e39-a740-d090633aefe9\") " pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.358152 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.358114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr74j\" (UniqueName: \"kubernetes.io/projected/657bb57a-8488-4e39-a740-d090633aefe9-kube-api-access-jr74j\") pod \"cert-manager-55b4d954d5-mnd5j\" (UID: \"657bb57a-8488-4e39-a740-d090633aefe9\") " pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.358334 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.358162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/657bb57a-8488-4e39-a740-d090633aefe9-bound-sa-token\") pod \"cert-manager-55b4d954d5-mnd5j\" (UID: \"657bb57a-8488-4e39-a740-d090633aefe9\") " pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.379751 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.379715    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/657bb57a-8488-4e39-a740-d090633aefe9-bound-sa-token\") pod \"cert-manager-55b4d954d5-mnd5j\" (UID: \"657bb57a-8488-4e39-a740-d090633aefe9\") " pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.380007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.379989    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr74j\" (UniqueName: \"kubernetes.io/projected/657bb57a-8488-4e39-a740-d090633aefe9-kube-api-access-jr74j\") pod \"cert-manager-55b4d954d5-mnd5j\" (UID: \"657bb57a-8488-4e39-a740-d090633aefe9\") " pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.549907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.549809    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mnd5j"
Jul 08 19:11:13.684862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.684829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mnd5j"]
Jul 08 19:11:13.687839 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:11:13.687809    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod657bb57a_8488_4e39_a740_d090633aefe9.slice/crio-3e16c0404cb2e9b9d138a63b53412ec304b4a8aa70d773e9b58b9ba5df0cc5da WatchSource:0}: Error finding container 3e16c0404cb2e9b9d138a63b53412ec304b4a8aa70d773e9b58b9ba5df0cc5da: Status 404 returned error can't find the container with id 3e16c0404cb2e9b9d138a63b53412ec304b4a8aa70d773e9b58b9ba5df0cc5da
Jul 08 19:11:13.689773 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:13.689755    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 19:11:14.354583 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:14.354546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mnd5j" event={"ID":"657bb57a-8488-4e39-a740-d090633aefe9","Type":"ContainerStarted","Data":"3e16c0404cb2e9b9d138a63b53412ec304b4a8aa70d773e9b58b9ba5df0cc5da"}
Jul 08 19:11:17.368483 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:17.368431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mnd5j" event={"ID":"657bb57a-8488-4e39-a740-d090633aefe9","Type":"ContainerStarted","Data":"c41b50df697410c63de92d63a8800dd8a7aa3448b350b489a7eef6090c192c8e"}
Jul 08 19:11:17.391862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:17.391804    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mnd5j" podStartSLOduration=1.183548073 podStartE2EDuration="4.391786276s" totalImagesPullingTime="3.208238203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:17.390652308 +0000 UTC m=+351.055214859" watchObservedRunningTime="2026-07-08 19:11:17.391786276 +0000 UTC m=+351.056348826"
Jul 08 19:11:37.632445 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.632409    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"]
Jul 08 19:11:37.635935 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.635919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.639335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.639310    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 08 19:11:37.639465 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.639450    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fl96f\""
Jul 08 19:11:37.639864 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.639847    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 08 19:11:37.640108 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.640069    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 08 19:11:37.640201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.640164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 08 19:11:37.658479 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.658451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"]
Jul 08 19:11:37.737554 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.737514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d43b7087-c441-4777-bc7b-f003624d0fcf-tmp\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.737762 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.737591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d43b7087-c441-4777-bc7b-f003624d0fcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.737762 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.737634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d43b7087-c441-4777-bc7b-f003624d0fcf-manifests\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.737762 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.737743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdpmg\" (UniqueName: \"kubernetes.io/projected/d43b7087-c441-4777-bc7b-f003624d0fcf-kube-api-access-rdpmg\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.737891 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.737792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d43b7087-c441-4777-bc7b-f003624d0fcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.839306 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.839268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d43b7087-c441-4777-bc7b-f003624d0fcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.839306 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.839307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d43b7087-c441-4777-bc7b-f003624d0fcf-manifests\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.839564 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.839337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdpmg\" (UniqueName: \"kubernetes.io/projected/d43b7087-c441-4777-bc7b-f003624d0fcf-kube-api-access-rdpmg\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.839564 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.839372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d43b7087-c441-4777-bc7b-f003624d0fcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.839564 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.839414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d43b7087-c441-4777-bc7b-f003624d0fcf-tmp\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.839790 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.839765    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d43b7087-c441-4777-bc7b-f003624d0fcf-manifests\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.841954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.841928    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d43b7087-c441-4777-bc7b-f003624d0fcf-tmp\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.842141 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.842118    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d43b7087-c441-4777-bc7b-f003624d0fcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.842191 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.842150    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d43b7087-c441-4777-bc7b-f003624d0fcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.858060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.858026    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdpmg\" (UniqueName: \"kubernetes.io/projected/d43b7087-c441-4777-bc7b-f003624d0fcf-kube-api-access-rdpmg\") pod \"opendatahub-operator-controller-manager-6667fbd97f-g6v25\" (UID: \"d43b7087-c441-4777-bc7b-f003624d0fcf\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:37.947340 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:37.947239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:38.134582 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:38.134558    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"]
Jul 08 19:11:38.137143 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:11:38.137114    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43b7087_c441_4777_bc7b_f003624d0fcf.slice/crio-abeb003f67f8efe5ff5e71e4c0e36aa66c3612a5d75148a279b49077649d1028 WatchSource:0}: Error finding container abeb003f67f8efe5ff5e71e4c0e36aa66c3612a5d75148a279b49077649d1028: Status 404 returned error can't find the container with id abeb003f67f8efe5ff5e71e4c0e36aa66c3612a5d75148a279b49077649d1028
Jul 08 19:11:38.444034 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:38.443998    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25" event={"ID":"d43b7087-c441-4777-bc7b-f003624d0fcf","Type":"ContainerStarted","Data":"abeb003f67f8efe5ff5e71e4c0e36aa66c3612a5d75148a279b49077649d1028"}
Jul 08 19:11:42.461342 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:42.461306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25" event={"ID":"d43b7087-c441-4777-bc7b-f003624d0fcf","Type":"ContainerStarted","Data":"0bab67e1564d8117da7c1e475b407c624c67a58bbbd2e74eca020eebd3177e77"}
Jul 08 19:11:43.467122 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:43.467073    2563 generic.go:356] "Generic (PLEG): container finished" podID="d43b7087-c441-4777-bc7b-f003624d0fcf" containerID="0bab67e1564d8117da7c1e475b407c624c67a58bbbd2e74eca020eebd3177e77" exitCode=0
Jul 08 19:11:43.467579 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:43.467130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25" event={"ID":"d43b7087-c441-4777-bc7b-f003624d0fcf","Type":"ContainerDied","Data":"0bab67e1564d8117da7c1e475b407c624c67a58bbbd2e74eca020eebd3177e77"}
Jul 08 19:11:44.473601 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:44.473562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25" event={"ID":"d43b7087-c441-4777-bc7b-f003624d0fcf","Type":"ContainerStarted","Data":"85db41227a00526b44a3f86972352e50c0e67b6730f9733f2d1399b37de14d4f"}
Jul 08 19:11:44.474110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:44.473681    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:55.480420 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:55.480385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25"
Jul 08 19:11:55.523490 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:55.523436    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-g6v25" podStartSLOduration=13.678868767 podStartE2EDuration="18.523418127s" totalImagesPullingTime="3.78288839s" totalInitContainerRuntime="1.06166097s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:44.523109252 +0000 UTC m=+378.187671794" watchObservedRunningTime="2026-07-08 19:11:55.523418127 +0000 UTC m=+389.187980679"
Jul 08 19:11:59.206350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.206308    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"]
Jul 08 19:11:59.208648 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.208630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.222767 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.222741    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 19:11:59.222926 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.222791    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 08 19:11:59.223870 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.223847    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 19:11:59.223989 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.223877    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 08 19:11:59.238690 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.238655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"]
Jul 08 19:11:59.328908 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.328876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6644d20a-3b8d-4677-ab0e-30677ba8e54d-tls-certs\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.328908 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.328918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s2q9\" (UniqueName: \"kubernetes.io/projected/6644d20a-3b8d-4677-ab0e-30677ba8e54d-kube-api-access-2s2q9\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.329148 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.328996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6644d20a-3b8d-4677-ab0e-30677ba8e54d-tmp\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.430282 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.430240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6644d20a-3b8d-4677-ab0e-30677ba8e54d-tls-certs\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.430483 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.430293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s2q9\" (UniqueName: \"kubernetes.io/projected/6644d20a-3b8d-4677-ab0e-30677ba8e54d-kube-api-access-2s2q9\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.430483 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.430339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6644d20a-3b8d-4677-ab0e-30677ba8e54d-tmp\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.432909 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.432880    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6644d20a-3b8d-4677-ab0e-30677ba8e54d-tmp\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.433130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.433109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6644d20a-3b8d-4677-ab0e-30677ba8e54d-tls-certs\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.458644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.458564    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s2q9\" (UniqueName: \"kubernetes.io/projected/6644d20a-3b8d-4677-ab0e-30677ba8e54d-kube-api-access-2s2q9\") pod \"kube-auth-proxy-565878cbdc-2j55g\" (UID: \"6644d20a-3b8d-4677-ab0e-30677ba8e54d\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.517962 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.517924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"
Jul 08 19:11:59.716765 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:11:59.716679    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565878cbdc-2j55g"]
Jul 08 19:11:59.724355 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:11:59.724319    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6644d20a_3b8d_4677_ab0e_30677ba8e54d.slice/crio-81186037a9dff872ebbfa1bcfbabdd7b3cf054a8902323924d6b1ca45e0d9a62 WatchSource:0}: Error finding container 81186037a9dff872ebbfa1bcfbabdd7b3cf054a8902323924d6b1ca45e0d9a62: Status 404 returned error can't find the container with id 81186037a9dff872ebbfa1bcfbabdd7b3cf054a8902323924d6b1ca45e0d9a62
Jul 08 19:12:00.537428 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:00.537390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g" event={"ID":"6644d20a-3b8d-4677-ab0e-30677ba8e54d","Type":"ContainerStarted","Data":"81186037a9dff872ebbfa1bcfbabdd7b3cf054a8902323924d6b1ca45e0d9a62"}
Jul 08 19:12:02.651630 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.651597    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"]
Jul 08 19:12:02.654602 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.654579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.661006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.660968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 08 19:12:02.661191 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.660967    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 08 19:12:02.679955 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.679917    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 08 19:12:02.680197 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.680176    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 19:12:02.680532 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.680491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kbxdf\""
Jul 08 19:12:02.708015 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.707972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"]
Jul 08 19:12:02.761064 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-cacerts\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.761064 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.761340 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5tcc\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-kube-api-access-n5tcc\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.761340 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-token\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.761340 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4a0aea-0d62-46a5-a150-0b0980a13171-local-certs\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.761478 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.761478 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.761388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862729 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862729 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-cacerts\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5tcc\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-kube-api-access-n5tcc\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-token\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.862981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.862918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4a0aea-0d62-46a5-a150-0b0980a13171-local-certs\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.863607 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.863554    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.865835 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.865805    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4a0aea-0d62-46a5-a150-0b0980a13171-local-certs\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.865957 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.865858    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.866184 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.866164    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-cacerts\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.866277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.866204    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.883876 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.883841    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5tcc\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-kube-api-access-n5tcc\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.892718 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.892691    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-token\") pod \"istiod-openshift-gateway-7574459656-zzgs6\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:02.967059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:02.966963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:03.154985 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:03.154902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"]
Jul 08 19:12:03.772923 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:12:03.772889    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4a0aea_0d62_46a5_a150_0b0980a13171.slice/crio-c7447b763176d0218c75f0cc4b24d3fb5a378abd77ad0e02d05cd0729a58a721 WatchSource:0}: Error finding container c7447b763176d0218c75f0cc4b24d3fb5a378abd77ad0e02d05cd0729a58a721: Status 404 returned error can't find the container with id c7447b763176d0218c75f0cc4b24d3fb5a378abd77ad0e02d05cd0729a58a721
Jul 08 19:12:04.562774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:04.562731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g" event={"ID":"6644d20a-3b8d-4677-ab0e-30677ba8e54d","Type":"ContainerStarted","Data":"541216cb6c68f692d2283a474d770ad996801a014b8d9ad140d175dba4929267"}
Jul 08 19:12:04.564579 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:04.564542    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" event={"ID":"7d4a0aea-0d62-46a5-a150-0b0980a13171","Type":"ContainerStarted","Data":"c7447b763176d0218c75f0cc4b24d3fb5a378abd77ad0e02d05cd0729a58a721"}
Jul 08 19:12:04.608483 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:04.608426    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-565878cbdc-2j55g" podStartSLOduration=1.5286325 podStartE2EDuration="5.608408779s" totalImagesPullingTime="4.079776279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:04.606454943 +0000 UTC m=+398.271017496" watchObservedRunningTime="2026-07-08 19:12:04.608408779 +0000 UTC m=+398.272971329"
Jul 08 19:12:06.243692 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:06.243644    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:12:06.243957 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:06.243730    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:12:06.583286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:06.583192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" event={"ID":"7d4a0aea-0d62-46a5-a150-0b0980a13171","Type":"ContainerStarted","Data":"9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395"}
Jul 08 19:12:06.583432 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:06.583345    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:06.643103 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:06.643020    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" podStartSLOduration=2.174541549 podStartE2EDuration="4.642998031s" totalImagesPullingTime="2.468456482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:06.64212057 +0000 UTC m=+400.306683118" watchObservedRunningTime="2026-07-08 19:12:06.642998031 +0000 UTC m=+400.307560582"
Jul 08 19:12:07.589330 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:07.589295    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-zzgs6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 19:12:07.589877 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:07.589363    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" podUID="7d4a0aea-0d62-46a5-a150-0b0980a13171" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 19:12:09.438453 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.438418    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"]
Jul 08 19:12:09.440723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.440707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.486979 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.486943    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"]
Jul 08 19:12:09.624873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.624829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9eba5102-c998-4dec-ae2b-237d3534f892-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.624873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.624878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htgq4\" (UniqueName: \"kubernetes.io/projected/9eba5102-c998-4dec-ae2b-237d3534f892-kube-api-access-htgq4\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.625164 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.624960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eba5102-c998-4dec-ae2b-237d3534f892-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.625164 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.624997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.625164 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.625023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.625164 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.625043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.625329 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.625157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eba5102-c998-4dec-ae2b-237d3534f892-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eba5102-c998-4dec-ae2b-237d3534f892-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726547 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eba5102-c998-4dec-ae2b-237d3534f892-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9eba5102-c998-4dec-ae2b-237d3534f892-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.726890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.726656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htgq4\" (UniqueName: \"kubernetes.io/projected/9eba5102-c998-4dec-ae2b-237d3534f892-kube-api-access-htgq4\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.727752 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.727719    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9eba5102-c998-4dec-ae2b-237d3534f892-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.729234 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.729203    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.729234 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.729225    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eba5102-c998-4dec-ae2b-237d3534f892-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.729378 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.729336    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.729378 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.729347    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9eba5102-c998-4dec-ae2b-237d3534f892-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.754236 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.754206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eba5102-c998-4dec-ae2b-237d3534f892-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:09.766992 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:09.766952    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htgq4\" (UniqueName: \"kubernetes.io/projected/9eba5102-c998-4dec-ae2b-237d3534f892-kube-api-access-htgq4\") pod \"istiod-openshift-gateway-94bb8fbfd-r8wsd\" (UID: \"9eba5102-c998-4dec-ae2b-237d3534f892\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:10.049950 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.049853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:10.267441 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.267249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"]
Jul 08 19:12:10.270055 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:12:10.270025    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eba5102_c998_4dec_ae2b_237d3534f892.slice/crio-26acf6be10b3a8bcc29c537b4a8ff52d1fc58dcb6dfce4b221901aea571556ac WatchSource:0}: Error finding container 26acf6be10b3a8bcc29c537b4a8ff52d1fc58dcb6dfce4b221901aea571556ac: Status 404 returned error can't find the container with id 26acf6be10b3a8bcc29c537b4a8ff52d1fc58dcb6dfce4b221901aea571556ac
Jul 08 19:12:10.273255 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.273218    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:12:10.273350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.273282    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:12:10.588890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.588802    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:10.603238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.603174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd" event={"ID":"9eba5102-c998-4dec-ae2b-237d3534f892","Type":"ContainerStarted","Data":"62121e89166838a7a766ed0a1af68996ac48757bcb78c41fb279dec1920555e1"}
Jul 08 19:12:10.603238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.603239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd" event={"ID":"9eba5102-c998-4dec-ae2b-237d3534f892","Type":"ContainerStarted","Data":"26acf6be10b3a8bcc29c537b4a8ff52d1fc58dcb6dfce4b221901aea571556ac"}
Jul 08 19:12:10.603644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.603288    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:10.699565 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:10.699503    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd" podStartSLOduration=1.699482744 podStartE2EDuration="1.699482744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:10.698196539 +0000 UTC m=+404.362759091" watchObservedRunningTime="2026-07-08 19:12:10.699482744 +0000 UTC m=+404.364045294"
Jul 08 19:12:11.610135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:11.610095    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-r8wsd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 19:12:11.610518 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:11.610152    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd" podUID="9eba5102-c998-4dec-ae2b-237d3534f892" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 19:12:14.609588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:14.609555    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r8wsd"
Jul 08 19:12:14.863039 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:14.862951    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"]
Jul 08 19:12:14.863205 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:14.863153    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" podUID="7d4a0aea-0d62-46a5-a150-0b0980a13171" containerName="discovery" containerID="cri-o://9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395" gracePeriod=30
Jul 08 19:12:15.111459 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.111435    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:15.175947 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.175861    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-token\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.175947 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.175907    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-kubeconfig\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.176216 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.175962    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4a0aea-0d62-46a5-a150-0b0980a13171-local-certs\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.176216 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.176000    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-cacerts\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.176216 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.176035    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-ca-configmap\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.176216 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.176056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-dns-cert\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.176216 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.176127    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5tcc\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-kube-api-access-n5tcc\") pod \"7d4a0aea-0d62-46a5-a150-0b0980a13171\" (UID: \"7d4a0aea-0d62-46a5-a150-0b0980a13171\") "
Jul 08 19:12:15.176565 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.176515    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:12:15.179016 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.178973    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-kube-api-access-n5tcc" (OuterVolumeSpecName: "kube-api-access-n5tcc") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "kube-api-access-n5tcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:12:15.179495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.179469    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:12:15.179591 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.179512    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:12:15.179591 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.179568    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-cacerts" (OuterVolumeSpecName: "cacerts") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:12:15.179752 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.179731    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4a0aea-0d62-46a5-a150-0b0980a13171-local-certs" (OuterVolumeSpecName: "local-certs") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:12:15.179868 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.179848    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7d4a0aea-0d62-46a5-a150-0b0980a13171" (UID: "7d4a0aea-0d62-46a5-a150-0b0980a13171"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:12:15.277298 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277265    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-ca-configmap\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.277298 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277298    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-csr-dns-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.277298 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277311    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5tcc\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-kube-api-access-n5tcc\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.277618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277324    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.277618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277337    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-istio-kubeconfig\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.277618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277348    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4a0aea-0d62-46a5-a150-0b0980a13171-local-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.277618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.277358    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d4a0aea-0d62-46a5-a150-0b0980a13171-cacerts\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:12:15.626404 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.626369    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d4a0aea-0d62-46a5-a150-0b0980a13171" containerID="9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395" exitCode=0
Jul 08 19:12:15.626832 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.626411    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"
Jul 08 19:12:15.626832 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.626442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" event={"ID":"7d4a0aea-0d62-46a5-a150-0b0980a13171","Type":"ContainerDied","Data":"9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395"}
Jul 08 19:12:15.626832 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.626497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6" event={"ID":"7d4a0aea-0d62-46a5-a150-0b0980a13171","Type":"ContainerDied","Data":"c7447b763176d0218c75f0cc4b24d3fb5a378abd77ad0e02d05cd0729a58a721"}
Jul 08 19:12:15.626832 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.626519    2563 scope.go:117] "RemoveContainer" containerID="9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395"
Jul 08 19:12:15.635884 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.635863    2563 scope.go:117] "RemoveContainer" containerID="9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395"
Jul 08 19:12:15.636278 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:12:15.636255    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395\": container with ID starting with 9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395 not found: ID does not exist" containerID="9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395"
Jul 08 19:12:15.636351 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.636291    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395"} err="failed to get container status \"9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395\": rpc error: code = NotFound desc = could not find container \"9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395\": container with ID starting with 9bc5249474daea78dd753cf88c5ef8bb67a91241585ad916158c6674bf3a1395 not found: ID does not exist"
Jul 08 19:12:15.749577 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.747227    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"]
Jul 08 19:12:15.749577 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.747972    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d4a0aea-0d62-46a5-a150-0b0980a13171" containerName="discovery"
Jul 08 19:12:15.749577 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.747990    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d4a0aea-0d62-46a5-a150-0b0980a13171" containerName="discovery"
Jul 08 19:12:15.749577 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.748160    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d4a0aea-0d62-46a5-a150-0b0980a13171" containerName="discovery"
Jul 08 19:12:15.752044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.752008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.775099 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.775048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wb5d2\""
Jul 08 19:12:15.775318 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.775248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 08 19:12:15.781044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.781018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9z4v\" (UniqueName: \"kubernetes.io/projected/fea46e71-eb0d-4ad6-96a9-94be06d0d4dd-kube-api-access-m9z4v\") pod \"kserve-controller-manager-d6f4b65b8-2frh5\" (UID: \"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.781197 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.781056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fea46e71-eb0d-4ad6-96a9-94be06d0d4dd-cert\") pod \"kserve-controller-manager-d6f4b65b8-2frh5\" (UID: \"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.836239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.836203    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"]
Jul 08 19:12:15.881876 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.881788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9z4v\" (UniqueName: \"kubernetes.io/projected/fea46e71-eb0d-4ad6-96a9-94be06d0d4dd-kube-api-access-m9z4v\") pod \"kserve-controller-manager-d6f4b65b8-2frh5\" (UID: \"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.881876 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.881834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fea46e71-eb0d-4ad6-96a9-94be06d0d4dd-cert\") pod \"kserve-controller-manager-d6f4b65b8-2frh5\" (UID: \"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.884485 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.884454    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fea46e71-eb0d-4ad6-96a9-94be06d0d4dd-cert\") pod \"kserve-controller-manager-d6f4b65b8-2frh5\" (UID: \"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.927701 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.927667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"]
Jul 08 19:12:15.929183 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.929152    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9z4v\" (UniqueName: \"kubernetes.io/projected/fea46e71-eb0d-4ad6-96a9-94be06d0d4dd-kube-api-access-m9z4v\") pod \"kserve-controller-manager-d6f4b65b8-2frh5\" (UID: \"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:15.929309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:15.929222    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zzgs6"]
Jul 08 19:12:16.061823 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:16.061784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:16.307663 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:16.307512    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"]
Jul 08 19:12:16.309921 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:12:16.309898    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea46e71_eb0d_4ad6_96a9_94be06d0d4dd.slice/crio-1747dcc95eda6664ed7b97374985dc60ccc1b868cc2b9135b0bc52e474e21cee WatchSource:0}: Error finding container 1747dcc95eda6664ed7b97374985dc60ccc1b868cc2b9135b0bc52e474e21cee: Status 404 returned error can't find the container with id 1747dcc95eda6664ed7b97374985dc60ccc1b868cc2b9135b0bc52e474e21cee
Jul 08 19:12:16.632902 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:16.632870    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5" event={"ID":"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd","Type":"ContainerStarted","Data":"1747dcc95eda6664ed7b97374985dc60ccc1b868cc2b9135b0bc52e474e21cee"}
Jul 08 19:12:16.929873 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:16.929771    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d4a0aea-0d62-46a5-a150-0b0980a13171" path="/var/lib/kubelet/pods/7d4a0aea-0d62-46a5-a150-0b0980a13171/volumes"
Jul 08 19:12:19.649225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:19.649189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5" event={"ID":"fea46e71-eb0d-4ad6-96a9-94be06d0d4dd","Type":"ContainerStarted","Data":"5287ebcf35309a7516a53f2409821e3032c9d58fb2c072d42428b303b1b1fd07"}
Jul 08 19:12:19.649655 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:19.649304    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:12:19.738780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:19.738663    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5" podStartSLOduration=1.569999099 podStartE2EDuration="4.738643776s" totalImagesPullingTime="3.168644677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:19.733435061 +0000 UTC m=+413.397997615" watchObservedRunningTime="2026-07-08 19:12:19.738643776 +0000 UTC m=+413.403206327"
Jul 08 19:12:50.659864 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:12:50.659777    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2frh5"
Jul 08 19:13:00.210220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.210184    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4vlhz"]
Jul 08 19:13:00.213724 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.213704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:00.217929 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.217904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xzndg\""
Jul 08 19:13:00.218955 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.218922    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 19:13:00.219110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.218933    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 19:13:00.225998 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.225967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4vlhz"]
Jul 08 19:13:00.286477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.286437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv5rd\" (UniqueName: \"kubernetes.io/projected/fb0aa254-de61-4722-bdcd-8429b56125a1-kube-api-access-fv5rd\") pod \"kuadrant-operator-catalog-4vlhz\" (UID: \"fb0aa254-de61-4722-bdcd-8429b56125a1\") " pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:00.387655 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.387611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv5rd\" (UniqueName: \"kubernetes.io/projected/fb0aa254-de61-4722-bdcd-8429b56125a1-kube-api-access-fv5rd\") pod \"kuadrant-operator-catalog-4vlhz\" (UID: \"fb0aa254-de61-4722-bdcd-8429b56125a1\") " pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:00.401814 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.401782    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv5rd\" (UniqueName: \"kubernetes.io/projected/fb0aa254-de61-4722-bdcd-8429b56125a1-kube-api-access-fv5rd\") pod \"kuadrant-operator-catalog-4vlhz\" (UID: \"fb0aa254-de61-4722-bdcd-8429b56125a1\") " pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:00.522978 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.522878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:00.659306 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.659272    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4vlhz"]
Jul 08 19:13:00.662750 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:00.662721    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0aa254_de61_4722_bdcd_8429b56125a1.slice/crio-61726fc26f2d70ea07f1c9898bba28e89a8af69957be1e580d445bce0b108fc2 WatchSource:0}: Error finding container 61726fc26f2d70ea07f1c9898bba28e89a8af69957be1e580d445bce0b108fc2: Status 404 returned error can't find the container with id 61726fc26f2d70ea07f1c9898bba28e89a8af69957be1e580d445bce0b108fc2
Jul 08 19:13:00.817897 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:00.817801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz" event={"ID":"fb0aa254-de61-4722-bdcd-8429b56125a1","Type":"ContainerStarted","Data":"61726fc26f2d70ea07f1c9898bba28e89a8af69957be1e580d445bce0b108fc2"}
Jul 08 19:13:02.830927 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:02.830893    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz" event={"ID":"fb0aa254-de61-4722-bdcd-8429b56125a1","Type":"ContainerStarted","Data":"6fbcfdc1070105d20126f140c247d40ac079fd849627f7dde9f15e5ecc5ccb34"}
Jul 08 19:13:02.850464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:02.850405    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz" podStartSLOduration=0.83320623 podStartE2EDuration="2.8503872s" totalImagesPullingTime="2.01718097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:02.849349091 +0000 UTC m=+456.513911641" watchObservedRunningTime="2026-07-08 19:13:02.8503872 +0000 UTC m=+456.514949751"
Jul 08 19:13:10.523872 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:10.523819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:10.524481 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:10.523907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:10.550109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:10.550062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:10.886845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:10.886819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4vlhz"
Jul 08 19:13:20.007064 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.007018    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fdff7d686-wfmnt"]
Jul 08 19:13:20.009573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.009548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.015682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.015652    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 19:13:20.015843 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.015822    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 19:13:20.016850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.016825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-76gvt\""
Jul 08 19:13:20.017026 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.017007    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 19:13:20.018368 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.018352    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 19:13:20.019095 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.019061    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 19:13:20.019660 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.019641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 19:13:20.019740 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.019651    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 19:13:20.021426 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.021406    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 19:13:20.035836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.035800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdff7d686-wfmnt"]
Jul 08 19:13:20.067044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-config\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.067263 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-oauth-config\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.067263 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-trusted-ca-bundle\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.067263 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tljdr\" (UniqueName: \"kubernetes.io/projected/7ec56def-70d4-4264-a5c4-9bd8222a3014-kube-api-access-tljdr\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.067395 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-serving-cert\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.067449 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-service-ca\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.067500 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.067455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-oauth-serving-cert\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-config\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-oauth-config\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-trusted-ca-bundle\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tljdr\" (UniqueName: \"kubernetes.io/projected/7ec56def-70d4-4264-a5c4-9bd8222a3014-kube-api-access-tljdr\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-serving-cert\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-service-ca\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.168874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.168838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-oauth-serving-cert\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.169588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.169559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-config\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.169588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.169577    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-service-ca\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.169744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.169676    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-oauth-serving-cert\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.169984 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.169963    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ec56def-70d4-4264-a5c4-9bd8222a3014-trusted-ca-bundle\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.171451 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.171434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-oauth-config\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.171556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.171538    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ec56def-70d4-4264-a5c4-9bd8222a3014-console-serving-cert\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.180358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.180329    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tljdr\" (UniqueName: \"kubernetes.io/projected/7ec56def-70d4-4264-a5c4-9bd8222a3014-kube-api-access-tljdr\") pod \"console-fdff7d686-wfmnt\" (UID: \"7ec56def-70d4-4264-a5c4-9bd8222a3014\") " pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.319356 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.319249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:20.464628 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.464600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdff7d686-wfmnt"]
Jul 08 19:13:20.466374 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:20.466344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec56def_70d4_4264_a5c4_9bd8222a3014.slice/crio-ca45d67d0813f0055ad9ed6503275a494a0bea885beb39da4ea4f65c8c7a894b WatchSource:0}: Error finding container ca45d67d0813f0055ad9ed6503275a494a0bea885beb39da4ea4f65c8c7a894b: Status 404 returned error can't find the container with id ca45d67d0813f0055ad9ed6503275a494a0bea885beb39da4ea4f65c8c7a894b
Jul 08 19:13:20.911966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.911930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdff7d686-wfmnt" event={"ID":"7ec56def-70d4-4264-a5c4-9bd8222a3014","Type":"ContainerStarted","Data":"c74a330bb97233b68be86b499de3459f7d589677d05ee76db8c588a358a2d668"}
Jul 08 19:13:20.912257 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.911983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdff7d686-wfmnt" event={"ID":"7ec56def-70d4-4264-a5c4-9bd8222a3014","Type":"ContainerStarted","Data":"ca45d67d0813f0055ad9ed6503275a494a0bea885beb39da4ea4f65c8c7a894b"}
Jul 08 19:13:20.934996 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:20.934940    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fdff7d686-wfmnt" podStartSLOduration=1.9349229110000001 podStartE2EDuration="1.934922911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:20.932870909 +0000 UTC m=+474.597433476" watchObservedRunningTime="2026-07-08 19:13:20.934922911 +0000 UTC m=+474.599485536"
Jul 08 19:13:30.319644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:30.319597    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:30.320164 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:30.319668    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:30.324802 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:30.324776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:30.957032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:30.957003    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fdff7d686-wfmnt"
Jul 08 19:13:31.528317 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.528283    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"]
Jul 08 19:13:31.530937 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.530919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:31.533838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.533819    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r2wdq\""
Jul 08 19:13:31.533963 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.533863    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 08 19:13:31.544176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.544154    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"]
Jul 08 19:13:31.678545 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.678505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs459\" (UniqueName: \"kubernetes.io/projected/4827bb5e-cd1e-45fd-a6b0-56044d60aa1a-kube-api-access-fs459\") pod \"dns-operator-controller-manager-864dd78c4c-ksgt9\" (UID: \"4827bb5e-cd1e-45fd-a6b0-56044d60aa1a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:31.779315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.779228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs459\" (UniqueName: \"kubernetes.io/projected/4827bb5e-cd1e-45fd-a6b0-56044d60aa1a-kube-api-access-fs459\") pod \"dns-operator-controller-manager-864dd78c4c-ksgt9\" (UID: \"4827bb5e-cd1e-45fd-a6b0-56044d60aa1a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:31.788736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.788702    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs459\" (UniqueName: \"kubernetes.io/projected/4827bb5e-cd1e-45fd-a6b0-56044d60aa1a-kube-api-access-fs459\") pod \"dns-operator-controller-manager-864dd78c4c-ksgt9\" (UID: \"4827bb5e-cd1e-45fd-a6b0-56044d60aa1a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:31.841271 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.841233    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:31.987013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:31.986977    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"]
Jul 08 19:13:31.988796 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:31.988755    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4827bb5e_cd1e_45fd_a6b0_56044d60aa1a.slice/crio-76b70b4816e69662cfddc9b049b9218818966619030a5ea4f207e678b8bcd21d WatchSource:0}: Error finding container 76b70b4816e69662cfddc9b049b9218818966619030a5ea4f207e678b8bcd21d: Status 404 returned error can't find the container with id 76b70b4816e69662cfddc9b049b9218818966619030a5ea4f207e678b8bcd21d
Jul 08 19:13:32.964780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:32.964739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9" event={"ID":"4827bb5e-cd1e-45fd-a6b0-56044d60aa1a","Type":"ContainerStarted","Data":"76b70b4816e69662cfddc9b049b9218818966619030a5ea4f207e678b8bcd21d"}
Jul 08 19:13:34.974160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:34.974121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9" event={"ID":"4827bb5e-cd1e-45fd-a6b0-56044d60aa1a","Type":"ContainerStarted","Data":"44dd4c7606362bdca77ea19eb50ea4a79788c6de639d283fdd0d2f036f9e4fda"}
Jul 08 19:13:34.974571 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:34.974241    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:34.999776 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:34.999722    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9" podStartSLOduration=1.586660242 podStartE2EDuration="3.999703966s" totalImagesPullingTime="2.413043724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:34.998300488 +0000 UTC m=+488.662863039" watchObservedRunningTime="2026-07-08 19:13:34.999703966 +0000 UTC m=+488.664266516"
Jul 08 19:13:36.983049 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:36.983005    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"]
Jul 08 19:13:37.000288 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.000249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.006305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.006273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qjbt8\""
Jul 08 19:13:37.012120 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.012073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"]
Jul 08 19:13:37.126858 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.126822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vh94\" (UniqueName: \"kubernetes.io/projected/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-kube-api-access-6vh94\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.127052 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.126872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.228228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.228187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vh94\" (UniqueName: \"kubernetes.io/projected/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-kube-api-access-6vh94\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.228228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.228240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.228631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.228613    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.240989 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.240907    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vh94\" (UniqueName: \"kubernetes.io/projected/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-kube-api-access-6vh94\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.312440 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.312405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:37.455118 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.455071    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"]
Jul 08 19:13:37.457648 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:37.457614    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4538a7_5747_4aee_8e4e_9d460a5dc8ce.slice/crio-f87c4f4c08874dc9c59b069cb64c39d7b7ea0499b128123215ff8ee5a7f512bd WatchSource:0}: Error finding container f87c4f4c08874dc9c59b069cb64c39d7b7ea0499b128123215ff8ee5a7f512bd: Status 404 returned error can't find the container with id f87c4f4c08874dc9c59b069cb64c39d7b7ea0499b128123215ff8ee5a7f512bd
Jul 08 19:13:37.989068 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:37.989028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw" event={"ID":"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce","Type":"ContainerStarted","Data":"f87c4f4c08874dc9c59b069cb64c39d7b7ea0499b128123215ff8ee5a7f512bd"}
Jul 08 19:13:43.204859 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.204824    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"]
Jul 08 19:13:43.208215 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.208167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:43.212155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.212070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nx8dx\""
Jul 08 19:13:43.226494 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.226461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"]
Jul 08 19:13:43.285559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.285510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28hvl\" (UniqueName: \"kubernetes.io/projected/01660ab3-4166-4a08-9cb0-7da02b2aebd2-kube-api-access-28hvl\") pod \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" (UID: \"01660ab3-4166-4a08-9cb0-7da02b2aebd2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:43.386671 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.386627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28hvl\" (UniqueName: \"kubernetes.io/projected/01660ab3-4166-4a08-9cb0-7da02b2aebd2-kube-api-access-28hvl\") pod \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" (UID: \"01660ab3-4166-4a08-9cb0-7da02b2aebd2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:43.414405 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.414369    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28hvl\" (UniqueName: \"kubernetes.io/projected/01660ab3-4166-4a08-9cb0-7da02b2aebd2-kube-api-access-28hvl\") pod \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" (UID: \"01660ab3-4166-4a08-9cb0-7da02b2aebd2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:43.522285 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.522194    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:43.675958 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:43.675887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"]
Jul 08 19:13:43.678831 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:43.678743    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01660ab3_4166_4a08_9cb0_7da02b2aebd2.slice/crio-cafc7850a0914c504ace434b0fbda25b280a2236f850c4cbe3d0c6a846e3196e WatchSource:0}: Error finding container cafc7850a0914c504ace434b0fbda25b280a2236f850c4cbe3d0c6a846e3196e: Status 404 returned error can't find the container with id cafc7850a0914c504ace434b0fbda25b280a2236f850c4cbe3d0c6a846e3196e
Jul 08 19:13:44.019364 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:44.019332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt" event={"ID":"01660ab3-4166-4a08-9cb0-7da02b2aebd2","Type":"ContainerStarted","Data":"cafc7850a0914c504ace434b0fbda25b280a2236f850c4cbe3d0c6a846e3196e"}
Jul 08 19:13:44.020845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:44.020821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw" event={"ID":"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce","Type":"ContainerStarted","Data":"1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b"}
Jul 08 19:13:44.020981 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:44.020947    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:44.066834 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:44.066775    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw" podStartSLOduration=2.239255981 podStartE2EDuration="8.066758301s" totalImagesPullingTime="5.82750232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:44.066621566 +0000 UTC m=+497.731184099" watchObservedRunningTime="2026-07-08 19:13:44.066758301 +0000 UTC m=+497.731320852"
Jul 08 19:13:45.980940 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:45.980905    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ksgt9"
Jul 08 19:13:47.035511 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:47.035472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt" event={"ID":"01660ab3-4166-4a08-9cb0-7da02b2aebd2","Type":"ContainerStarted","Data":"01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023"}
Jul 08 19:13:47.035904 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:47.035556    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:47.091434 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:47.091377    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt" podStartSLOduration=1.754438613 podStartE2EDuration="4.091358116s" totalImagesPullingTime="2.336919503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:47.080170247 +0000 UTC m=+500.744732799" watchObservedRunningTime="2026-07-08 19:13:47.091358116 +0000 UTC m=+500.755920666"
Jul 08 19:13:55.028749 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:55.028709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:56.893884 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:56.893847    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"]
Jul 08 19:13:56.896711 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:56.896685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:56.913994 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:56.913964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"]
Jul 08 19:13:56.986802 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:56.986758    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"]
Jul 08 19:13:56.986992 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:56.986969    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" containerName="manager" containerID="cri-o://1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b" gracePeriod=2
Jul 08 19:13:57.006107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.006060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6ch6\" (UniqueName: \"kubernetes.io/projected/afd5ea5f-adf8-4615-9014-bbe17fcece5c-kube-api-access-r6ch6\") pod \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" (UID: \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.006289 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.006260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd5ea5f-adf8-4615-9014-bbe17fcece5c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" (UID: \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.012434 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.012397    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"]
Jul 08 19:13:57.024147 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.024113    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"]
Jul 08 19:13:57.024432 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.024413    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-r6ch6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c"
Jul 08 19:13:57.045738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.045705    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"]
Jul 08 19:13:57.046110 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.046078    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" containerName="manager"
Jul 08 19:13:57.046158 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.046112    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" containerName="manager"
Jul 08 19:13:57.046192 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.046178    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" containerName="manager"
Jul 08 19:13:57.048157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.048131    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.054243 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.054210    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"]
Jul 08 19:13:57.055050 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.055009    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.069094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.069057    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"]
Jul 08 19:13:57.069292 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.069270    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" containerName="manager" containerID="cri-o://01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023" gracePeriod=2
Jul 08 19:13:57.071870 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.071842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.083168 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.083137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.098783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.098736    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"]
Jul 08 19:13:57.100976 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.100944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"]
Jul 08 19:13:57.107807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.107775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6ch6\" (UniqueName: \"kubernetes.io/projected/afd5ea5f-adf8-4615-9014-bbe17fcece5c-kube-api-access-r6ch6\") pod \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" (UID: \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.107906 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.107837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5vnpb\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.107906 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.107892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95hk9\" (UniqueName: \"kubernetes.io/projected/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-kube-api-access-95hk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5vnpb\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.108001 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.107977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd5ea5f-adf8-4615-9014-bbe17fcece5c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" (UID: \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.108382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.108365    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd5ea5f-adf8-4615-9014-bbe17fcece5c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" (UID: \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.114006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.113383    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"]
Jul 08 19:13:57.114006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.113896    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" containerName="manager"
Jul 08 19:13:57.114006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.113918    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" containerName="manager"
Jul 08 19:13:57.114006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.114000    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" containerName="manager"
Jul 08 19:13:57.114592 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.114552    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.116145 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.116121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.117550 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.117527    2563 projected.go:196] Error preparing data for projected volume kube-api-access-r6ch6 for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-tp9jm" not found
Jul 08 19:13:57.117678 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.117624    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afd5ea5f-adf8-4615-9014-bbe17fcece5c-kube-api-access-r6ch6 podName:afd5ea5f-adf8-4615-9014-bbe17fcece5c nodeName:}" failed. No retries permitted until 2026-07-08 19:13:57.617594754 +0000 UTC m=+511.282157300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r6ch6" (UniqueName: "kubernetes.io/projected/afd5ea5f-adf8-4615-9014-bbe17fcece5c-kube-api-access-r6ch6") pod "kuadrant-operator-controller-manager-86878c769f-tp9jm" (UID: "afd5ea5f-adf8-4615-9014-bbe17fcece5c") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-tp9jm" not found
Jul 08 19:13:57.117678 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.117642    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.120696 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.120668    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.122987 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.122955    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.150840 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.150765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"]
Jul 08 19:13:57.152738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.152710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.182145 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.182075    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.184756 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.184718    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.188004 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.187962    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.190653 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.190601    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.193401 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.193370    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.208988 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.208956    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd5ea5f-adf8-4615-9014-bbe17fcece5c-extensions-socket-volume\") pod \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\" (UID: \"afd5ea5f-adf8-4615-9014-bbe17fcece5c\") "
Jul 08 19:13:57.209210 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95hk9\" (UniqueName: \"kubernetes.io/projected/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-kube-api-access-95hk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5vnpb\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.209210 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kclnt\" (UniqueName: \"kubernetes.io/projected/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-kube-api-access-kclnt\") pod \"kuadrant-operator-controller-manager-86878c769f-fqb7r\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.209210 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fqb7r\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.209436 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209256    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd5ea5f-adf8-4615-9014-bbe17fcece5c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "afd5ea5f-adf8-4615-9014-bbe17fcece5c" (UID: "afd5ea5f-adf8-4615-9014-bbe17fcece5c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:13:57.209436 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5vnpb\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.209436 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209344    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6ch6\" (UniqueName: \"kubernetes.io/projected/afd5ea5f-adf8-4615-9014-bbe17fcece5c-kube-api-access-r6ch6\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:13:57.209436 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209359    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd5ea5f-adf8-4615-9014-bbe17fcece5c-extensions-socket-volume\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:13:57.209696 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.209674    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5vnpb\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.232871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.232834    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95hk9\" (UniqueName: \"kubernetes.io/projected/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-kube-api-access-95hk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5vnpb\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.309837 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.309801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kclnt\" (UniqueName: \"kubernetes.io/projected/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-kube-api-access-kclnt\") pod \"kuadrant-operator-controller-manager-86878c769f-fqb7r\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.309837 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.309842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fqb7r\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.310225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.310206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fqb7r\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.318278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.318252    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.320731 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.320710    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.324827 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.324801    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.327926 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.327903    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.330235 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.330203    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.330661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.330638    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kclnt\" (UniqueName: \"kubernetes.io/projected/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-kube-api-access-kclnt\") pod \"kuadrant-operator-controller-manager-86878c769f-fqb7r\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.335212 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.335181    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:57.338466 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.338439    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:57.341160 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:57.341136    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:57.372152 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.372105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:13:57.410470 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.410369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vh94\" (UniqueName: \"kubernetes.io/projected/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-kube-api-access-6vh94\") pod \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") "
Jul 08 19:13:57.410470 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.410421    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28hvl\" (UniqueName: \"kubernetes.io/projected/01660ab3-4166-4a08-9cb0-7da02b2aebd2-kube-api-access-28hvl\") pod \"01660ab3-4166-4a08-9cb0-7da02b2aebd2\" (UID: \"01660ab3-4166-4a08-9cb0-7da02b2aebd2\") "
Jul 08 19:13:57.410470 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.410452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-extensions-socket-volume\") pod \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\" (UID: \"4b4538a7-5747-4aee-8e4e-9d460a5dc8ce\") "
Jul 08 19:13:57.411121 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.411073    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" (UID: "4b4538a7-5747-4aee-8e4e-9d460a5dc8ce"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:13:57.414201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.414167    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-kube-api-access-6vh94" (OuterVolumeSpecName: "kube-api-access-6vh94") pod "4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" (UID: "4b4538a7-5747-4aee-8e4e-9d460a5dc8ce"). InnerVolumeSpecName "kube-api-access-6vh94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:13:57.414201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.414194    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01660ab3-4166-4a08-9cb0-7da02b2aebd2-kube-api-access-28hvl" (OuterVolumeSpecName: "kube-api-access-28hvl") pod "01660ab3-4166-4a08-9cb0-7da02b2aebd2" (UID: "01660ab3-4166-4a08-9cb0-7da02b2aebd2"). InnerVolumeSpecName "kube-api-access-28hvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:13:57.461469 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.461437    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:57.511907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.511870    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vh94\" (UniqueName: \"kubernetes.io/projected/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-kube-api-access-6vh94\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:13:57.511907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.511900    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28hvl\" (UniqueName: \"kubernetes.io/projected/01660ab3-4166-4a08-9cb0-7da02b2aebd2-kube-api-access-28hvl\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:13:57.511907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.511913    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce-extensions-socket-volume\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:13:57.552924 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.552896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"]
Jul 08 19:13:57.555143 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:57.555114    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda53cfdf4_bf8a_437e_be73_5bc16c379bb9.slice/crio-31860d213d3faea4ef8c9539d5b8b3d2cc7f86b9c6bb8ffff58bba4211e42b3f WatchSource:0}: Error finding container 31860d213d3faea4ef8c9539d5b8b3d2cc7f86b9c6bb8ffff58bba4211e42b3f: Status 404 returned error can't find the container with id 31860d213d3faea4ef8c9539d5b8b3d2cc7f86b9c6bb8ffff58bba4211e42b3f
Jul 08 19:13:57.627206 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:57.627180    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"]
Jul 08 19:13:57.629361 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:13:57.629337    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a88e71_6b53_4ee0_9753_5c762aa4cb45.slice/crio-c123dfdd74a3cea8347570b1ef6eea61619891a46b9d182bc25e31697cfda142 WatchSource:0}: Error finding container c123dfdd74a3cea8347570b1ef6eea61619891a46b9d182bc25e31697cfda142: Status 404 returned error can't find the container with id c123dfdd74a3cea8347570b1ef6eea61619891a46b9d182bc25e31697cfda142
Jul 08 19:13:58.089528 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.089492    2563 generic.go:356] "Generic (PLEG): container finished" podID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" containerID="01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023" exitCode=0
Jul 08 19:13:58.089936 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.089540    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:58.090098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.089588    2563 scope.go:117] "RemoveContainer" containerID="01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023"
Jul 08 19:13:58.092621 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.092581    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:58.094711 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.094677    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:58.096755 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.096722    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:58.098360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.098263    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" containerID="1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b" exitCode=0
Jul 08 19:13:58.098360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.098332    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:58.100522 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.100494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r" event={"ID":"a5a88e71-6b53-4ee0-9753-5c762aa4cb45","Type":"ContainerStarted","Data":"11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6"}
Jul 08 19:13:58.100673 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.100607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r" event={"ID":"a5a88e71-6b53-4ee0-9753-5c762aa4cb45","Type":"ContainerStarted","Data":"c123dfdd74a3cea8347570b1ef6eea61619891a46b9d182bc25e31697cfda142"}
Jul 08 19:13:58.100673 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.100639    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:13:58.100828 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.100805    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:58.102919 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.102795    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:58.102919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.102903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb" event={"ID":"a53cfdf4-bf8a-437e-be73-5bc16c379bb9","Type":"ContainerStarted","Data":"dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d"}
Jul 08 19:13:58.103119 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.102924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:58.103119 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.102952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb" event={"ID":"a53cfdf4-bf8a-437e-be73-5bc16c379bb9","Type":"ContainerStarted","Data":"31860d213d3faea4ef8c9539d5b8b3d2cc7f86b9c6bb8ffff58bba4211e42b3f"}
Jul 08 19:13:58.105113 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.104935    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:58.107099 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.107058    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:58.108927 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.108911    2563 scope.go:117] "RemoveContainer" containerID="01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023"
Jul 08 19:13:58.109060 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.108938    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:58.109362 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.109337    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023\": container with ID starting with 01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023 not found: ID does not exist" containerID="01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023"
Jul 08 19:13:58.109427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.109374    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023"} err="failed to get container status \"01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023\": rpc error: code = NotFound desc = could not find container \"01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023\": container with ID starting with 01a21f25cef20c2e181ab02d9856859adc90953c5edbe987e61c68a0b8423023 not found: ID does not exist"
Jul 08 19:13:58.109427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.109395    2563 scope.go:117] "RemoveContainer" containerID="1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b"
Jul 08 19:13:58.119535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.119516    2563 scope.go:117] "RemoveContainer" containerID="1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b"
Jul 08 19:13:58.119845 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.119826    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b\": container with ID starting with 1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b not found: ID does not exist" containerID="1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b"
Jul 08 19:13:58.119898 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.119857    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b"} err="failed to get container status \"1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b\": rpc error: code = NotFound desc = could not find container \"1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b\": container with ID starting with 1e2c838523e07bf3a17c3d11da14285b89cc6e515a01d82a2b09fd7e92d4102b not found: ID does not exist"
Jul 08 19:13:58.127677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.127625    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb" podStartSLOduration=1.127611984 podStartE2EDuration="1.127611984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:58.12693198 +0000 UTC m=+511.791494531" watchObservedRunningTime="2026-07-08 19:13:58.127611984 +0000 UTC m=+511.792174534"
Jul 08 19:13:58.129066 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.129032    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:58.137794 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.137764    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4538a7_5747_4aee_8e4e_9d460a5dc8ce.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd5ea5f_adf8_4615_9014_bbe17fcece5c.slice\": RecentStats: unable to find data in memory cache]"
Jul 08 19:13:58.151222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.151174    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r" podStartSLOduration=1.151158288 podStartE2EDuration="1.151158288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:58.150884545 +0000 UTC m=+511.815447095" watchObservedRunningTime="2026-07-08 19:13:58.151158288 +0000 UTC m=+511.815720838"
Jul 08 19:13:58.153035 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.153002    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fgfgt\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fgfgt"
Jul 08 19:13:58.154984 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.154947    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-tp9jm\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tp9jm"
Jul 08 19:13:58.156998 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:13:58.156967    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7wnw\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7wnw"
Jul 08 19:13:58.929545 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.929508    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01660ab3-4166-4a08-9cb0-7da02b2aebd2" path="/var/lib/kubelet/pods/01660ab3-4166-4a08-9cb0-7da02b2aebd2/volumes"
Jul 08 19:13:58.929863 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.929850    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b4538a7-5747-4aee-8e4e-9d460a5dc8ce" path="/var/lib/kubelet/pods/4b4538a7-5747-4aee-8e4e-9d460a5dc8ce/volumes"
Jul 08 19:13:58.930154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:58.930141    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afd5ea5f-adf8-4615-9014-bbe17fcece5c" path="/var/lib/kubelet/pods/afd5ea5f-adf8-4615-9014-bbe17fcece5c/volumes"
Jul 08 19:13:59.110045 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:13:59.110004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:14:09.112771 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.112740    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:14:09.172634 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.172589    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"]
Jul 08 19:14:09.172945 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.172917    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb" podUID="a53cfdf4-bf8a-437e-be73-5bc16c379bb9" containerName="manager" containerID="cri-o://dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d" gracePeriod=10
Jul 08 19:14:09.175006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.174983    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:14:09.437071 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.437041    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:14:09.523059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.523029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-extensions-socket-volume\") pod \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") "
Jul 08 19:14:09.523059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.523064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95hk9\" (UniqueName: \"kubernetes.io/projected/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-kube-api-access-95hk9\") pod \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\" (UID: \"a53cfdf4-bf8a-437e-be73-5bc16c379bb9\") "
Jul 08 19:14:09.523513 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.523485    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a53cfdf4-bf8a-437e-be73-5bc16c379bb9" (UID: "a53cfdf4-bf8a-437e-be73-5bc16c379bb9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:14:09.525906 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.525859    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-kube-api-access-95hk9" (OuterVolumeSpecName: "kube-api-access-95hk9") pod "a53cfdf4-bf8a-437e-be73-5bc16c379bb9" (UID: "a53cfdf4-bf8a-437e-be73-5bc16c379bb9"). InnerVolumeSpecName "kube-api-access-95hk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:14:09.624666 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.624630    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-extensions-socket-volume\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:14:09.624666 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:09.624657    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95hk9\" (UniqueName: \"kubernetes.io/projected/a53cfdf4-bf8a-437e-be73-5bc16c379bb9-kube-api-access-95hk9\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:14:10.167858 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.167822    2563 generic.go:356] "Generic (PLEG): container finished" podID="a53cfdf4-bf8a-437e-be73-5bc16c379bb9" containerID="dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d" exitCode=0
Jul 08 19:14:10.168363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.167873    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"
Jul 08 19:14:10.168363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.167915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb" event={"ID":"a53cfdf4-bf8a-437e-be73-5bc16c379bb9","Type":"ContainerDied","Data":"dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d"}
Jul 08 19:14:10.168363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.167988    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb" event={"ID":"a53cfdf4-bf8a-437e-be73-5bc16c379bb9","Type":"ContainerDied","Data":"31860d213d3faea4ef8c9539d5b8b3d2cc7f86b9c6bb8ffff58bba4211e42b3f"}
Jul 08 19:14:10.168363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.168024    2563 scope.go:117] "RemoveContainer" containerID="dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d"
Jul 08 19:14:10.178071 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.177938    2563 scope.go:117] "RemoveContainer" containerID="dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d"
Jul 08 19:14:10.178297 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:14:10.178274    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d\": container with ID starting with dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d not found: ID does not exist" containerID="dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d"
Jul 08 19:14:10.178356 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.178306    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d"} err="failed to get container status \"dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d\": rpc error: code = NotFound desc = could not find container \"dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d\": container with ID starting with dc937bfb3f3c5cb4baf8db23f390eb61e3de8d943bb53fc05e36874d3fc4095d not found: ID does not exist"
Jul 08 19:14:10.194714 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.194682    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"]
Jul 08 19:14:10.199723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.199695    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5vnpb"]
Jul 08 19:14:10.930379 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:10.930349    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a53cfdf4-bf8a-437e-be73-5bc16c379bb9" path="/var/lib/kubelet/pods/a53cfdf4-bf8a-437e-be73-5bc16c379bb9/volumes"
Jul 08 19:14:25.583172 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.583064    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"]
Jul 08 19:14:25.583649 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.583588    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a53cfdf4-bf8a-437e-be73-5bc16c379bb9" containerName="manager"
Jul 08 19:14:25.583649 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.583604    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53cfdf4-bf8a-437e-be73-5bc16c379bb9" containerName="manager"
Jul 08 19:14:25.583736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.583677    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a53cfdf4-bf8a-437e-be73-5bc16c379bb9" containerName="manager"
Jul 08 19:14:25.587820 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.587796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.590765 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.590743    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vrjlv\""
Jul 08 19:14:25.600423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.600398    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"]
Jul 08 19:14:25.662727 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.662892 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.662892 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/566a6cc7-9156-43fe-a1fa-853634067c32-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.662892 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/566a6cc7-9156-43fe-a1fa-853634067c32-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.662995 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqhcq\" (UniqueName: \"kubernetes.io/projected/566a6cc7-9156-43fe-a1fa-853634067c32-kube-api-access-kqhcq\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.662995 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.662995 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.662986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.663098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.663034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/566a6cc7-9156-43fe-a1fa-853634067c32-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.663098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.663064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764405 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764405 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764647 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/566a6cc7-9156-43fe-a1fa-853634067c32-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764647 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764647 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764810 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764810 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/566a6cc7-9156-43fe-a1fa-853634067c32-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764810 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/566a6cc7-9156-43fe-a1fa-853634067c32-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764863    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqhcq\" (UniqueName: \"kubernetes.io/projected/566a6cc7-9156-43fe-a1fa-853634067c32-kube-api-access-kqhcq\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.764954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.764895    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.765240 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.765213    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.765365 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.765250    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.765365 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.765291    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/566a6cc7-9156-43fe-a1fa-853634067c32-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.767457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.767436    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/566a6cc7-9156-43fe-a1fa-853634067c32-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.767584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.767561    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/566a6cc7-9156-43fe-a1fa-853634067c32-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.778535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.778501    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqhcq\" (UniqueName: \"kubernetes.io/projected/566a6cc7-9156-43fe-a1fa-853634067c32-kube-api-access-kqhcq\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.778687 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.778664    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/566a6cc7-9156-43fe-a1fa-853634067c32-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9mvc4\" (UID: \"566a6cc7-9156-43fe-a1fa-853634067c32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:25.897788 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:25.897755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:26.052203 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:26.052178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"]
Jul 08 19:14:26.054304 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:14:26.054278    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566a6cc7_9156_43fe_a1fa_853634067c32.slice/crio-6b70bd9a6fdb606e00c93315c00b322970516abe841329f90b25e56463248a50 WatchSource:0}: Error finding container 6b70bd9a6fdb606e00c93315c00b322970516abe841329f90b25e56463248a50: Status 404 returned error can't find the container with id 6b70bd9a6fdb606e00c93315c00b322970516abe841329f90b25e56463248a50
Jul 08 19:14:26.247977 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:26.247887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" event={"ID":"566a6cc7-9156-43fe-a1fa-853634067c32","Type":"ContainerStarted","Data":"6b70bd9a6fdb606e00c93315c00b322970516abe841329f90b25e56463248a50"}
Jul 08 19:14:28.931194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:28.931155    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:14:28.931480 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:28.931228    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:14:28.931480 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:28.931257    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:14:29.264151 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:29.264040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" event={"ID":"566a6cc7-9156-43fe-a1fa-853634067c32","Type":"ContainerStarted","Data":"d8ed61d4501145b50bc78517ba73bd5d123ab27a8556bff6603bd691a2569e81"}
Jul 08 19:14:29.288459 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:29.288399    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" podStartSLOduration=1.414143582 podStartE2EDuration="4.288380221s" totalImagesPullingTime="2.874236639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:29.286628295 +0000 UTC m=+542.951190847" watchObservedRunningTime="2026-07-08 19:14:29.288380221 +0000 UTC m=+542.952942771"
Jul 08 19:14:29.898905 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:29.898869    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:29.903806 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:29.903777    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:30.268633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.268550    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:30.269559 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.269541    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:14:30.395051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.395011    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:30.398516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.398497    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.401236 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.401208    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 08 19:14:30.401530 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.401503    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vjhgt\""
Jul 08 19:14:30.405556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.405526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:30.490153 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.490111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:30.513207 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.513176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fxjj\" (UniqueName: \"kubernetes.io/projected/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-kube-api-access-8fxjj\") pod \"limitador-limitador-64c94fd64f-fvnqc\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.513383 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.513233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-config-file\") pod \"limitador-limitador-64c94fd64f-fvnqc\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.614202 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.614155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-config-file\") pod \"limitador-limitador-64c94fd64f-fvnqc\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.614392 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.614257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fxjj\" (UniqueName: \"kubernetes.io/projected/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-kube-api-access-8fxjj\") pod \"limitador-limitador-64c94fd64f-fvnqc\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.614897 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.614878    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-config-file\") pod \"limitador-limitador-64c94fd64f-fvnqc\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.623461 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.623426    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fxjj\" (UniqueName: \"kubernetes.io/projected/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-kube-api-access-8fxjj\") pod \"limitador-limitador-64c94fd64f-fvnqc\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.710544 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.710501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:30.854924 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:30.854887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:30.858217 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:14:30.858185    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010cfc77_6d1d_40eb_a1b7_7ed871f110fd.slice/crio-4482785804e07ecec1993e512ecce16deff5093ec5a241a10ca013ffa9057656 WatchSource:0}: Error finding container 4482785804e07ecec1993e512ecce16deff5093ec5a241a10ca013ffa9057656: Status 404 returned error can't find the container with id 4482785804e07ecec1993e512ecce16deff5093ec5a241a10ca013ffa9057656
Jul 08 19:14:31.274465 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:31.274430    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc" event={"ID":"010cfc77-6d1d-40eb-a1b7-7ed871f110fd","Type":"ContainerStarted","Data":"4482785804e07ecec1993e512ecce16deff5093ec5a241a10ca013ffa9057656"}
Jul 08 19:14:34.291349 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:34.291311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc" event={"ID":"010cfc77-6d1d-40eb-a1b7-7ed871f110fd","Type":"ContainerStarted","Data":"02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5"}
Jul 08 19:14:34.291815 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:34.291383    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:34.321007 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:34.320952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc" podStartSLOduration=1.513088019 podStartE2EDuration="4.320934642s" totalImagesPullingTime="2.807846623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:34.318406899 +0000 UTC m=+547.982969450" watchObservedRunningTime="2026-07-08 19:14:34.320934642 +0000 UTC m=+547.985497227"
Jul 08 19:14:45.297027 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:45.296989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:47.061573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.061535    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:47.062051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.061720    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc" podUID="010cfc77-6d1d-40eb-a1b7-7ed871f110fd" containerName="limitador" containerID="cri-o://02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5" gracePeriod=30
Jul 08 19:14:47.625818 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.625754    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:47.776717 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.776674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-config-file\") pod \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") "
Jul 08 19:14:47.776717 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.776727    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fxjj\" (UniqueName: \"kubernetes.io/projected/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-kube-api-access-8fxjj\") pod \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\" (UID: \"010cfc77-6d1d-40eb-a1b7-7ed871f110fd\") "
Jul 08 19:14:47.777130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.777070    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-config-file" (OuterVolumeSpecName: "config-file") pod "010cfc77-6d1d-40eb-a1b7-7ed871f110fd" (UID: "010cfc77-6d1d-40eb-a1b7-7ed871f110fd"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:14:47.779055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.779031    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-kube-api-access-8fxjj" (OuterVolumeSpecName: "kube-api-access-8fxjj") pod "010cfc77-6d1d-40eb-a1b7-7ed871f110fd" (UID: "010cfc77-6d1d-40eb-a1b7-7ed871f110fd"). InnerVolumeSpecName "kube-api-access-8fxjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:14:47.877560 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.877469    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-config-file\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:14:47.877560 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:47.877501    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fxjj\" (UniqueName: \"kubernetes.io/projected/010cfc77-6d1d-40eb-a1b7-7ed871f110fd-kube-api-access-8fxjj\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:14:48.361227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.361186    2563 generic.go:356] "Generic (PLEG): container finished" podID="010cfc77-6d1d-40eb-a1b7-7ed871f110fd" containerID="02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5" exitCode=0
Jul 08 19:14:48.361227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.361230    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"
Jul 08 19:14:48.361659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.361260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc" event={"ID":"010cfc77-6d1d-40eb-a1b7-7ed871f110fd","Type":"ContainerDied","Data":"02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5"}
Jul 08 19:14:48.361659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.361308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fvnqc" event={"ID":"010cfc77-6d1d-40eb-a1b7-7ed871f110fd","Type":"ContainerDied","Data":"4482785804e07ecec1993e512ecce16deff5093ec5a241a10ca013ffa9057656"}
Jul 08 19:14:48.361659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.361330    2563 scope.go:117] "RemoveContainer" containerID="02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5"
Jul 08 19:14:48.370790 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.370768    2563 scope.go:117] "RemoveContainer" containerID="02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5"
Jul 08 19:14:48.371072 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:14:48.371053    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5\": container with ID starting with 02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5 not found: ID does not exist" containerID="02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5"
Jul 08 19:14:48.371165 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.371107    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5"} err="failed to get container status \"02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5\": rpc error: code = NotFound desc = could not find container \"02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5\": container with ID starting with 02f597f7e6b393cf8e3f9b99740f2742e4390225c7814b9fd383886ef07ca3a5 not found: ID does not exist"
Jul 08 19:14:48.390721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.390671    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:48.392026 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.391999    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fvnqc"]
Jul 08 19:14:48.930394 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:48.930354    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="010cfc77-6d1d-40eb-a1b7-7ed871f110fd" path="/var/lib/kubelet/pods/010cfc77-6d1d-40eb-a1b7-7ed871f110fd/volumes"
Jul 08 19:14:52.146565 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.146529    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"]
Jul 08 19:14:52.146961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.146911    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010cfc77-6d1d-40eb-a1b7-7ed871f110fd" containerName="limitador"
Jul 08 19:14:52.146961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.146924    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="010cfc77-6d1d-40eb-a1b7-7ed871f110fd" containerName="limitador"
Jul 08 19:14:52.147032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.146985    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="010cfc77-6d1d-40eb-a1b7-7ed871f110fd" containerName="limitador"
Jul 08 19:14:52.151009 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.150986    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.154879 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.154859    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 08 19:14:52.156575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.156545    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vgbxx\""
Jul 08 19:14:52.156575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.156564    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 08 19:14:52.157450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.157373    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 08 19:14:52.163273 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.163249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"]
Jul 08 19:14:52.320755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.320712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgllt\" (UniqueName: \"kubernetes.io/projected/d6084d7a-acd4-423d-a5b3-ba29fc55d0ef-kube-api-access-xgllt\") pod \"postgres-69fb4594cf-4tmjr\" (UID: \"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.320909 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.320787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d6084d7a-acd4-423d-a5b3-ba29fc55d0ef-data\") pod \"postgres-69fb4594cf-4tmjr\" (UID: \"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.421384 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.421285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgllt\" (UniqueName: \"kubernetes.io/projected/d6084d7a-acd4-423d-a5b3-ba29fc55d0ef-kube-api-access-xgllt\") pod \"postgres-69fb4594cf-4tmjr\" (UID: \"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.421384 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.421375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d6084d7a-acd4-423d-a5b3-ba29fc55d0ef-data\") pod \"postgres-69fb4594cf-4tmjr\" (UID: \"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.421762 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.421739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d6084d7a-acd4-423d-a5b3-ba29fc55d0ef-data\") pod \"postgres-69fb4594cf-4tmjr\" (UID: \"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.431718 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.431683    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgllt\" (UniqueName: \"kubernetes.io/projected/d6084d7a-acd4-423d-a5b3-ba29fc55d0ef-kube-api-access-xgllt\") pod \"postgres-69fb4594cf-4tmjr\" (UID: \"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.460911 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.460869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:52.610155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:52.610008    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"]
Jul 08 19:14:52.612693 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:14:52.612648    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6084d7a_acd4_423d_a5b3_ba29fc55d0ef.slice/crio-ded5e0653fa6a1190f485f093e5186017bc21eb3837951193f2361347d118763 WatchSource:0}: Error finding container ded5e0653fa6a1190f485f093e5186017bc21eb3837951193f2361347d118763: Status 404 returned error can't find the container with id ded5e0653fa6a1190f485f093e5186017bc21eb3837951193f2361347d118763
Jul 08 19:14:53.391887 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:53.391808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr" event={"ID":"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef","Type":"ContainerStarted","Data":"ded5e0653fa6a1190f485f093e5186017bc21eb3837951193f2361347d118763"}
Jul 08 19:14:57.419072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:57.419033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr" event={"ID":"d6084d7a-acd4-423d-a5b3-ba29fc55d0ef","Type":"ContainerStarted","Data":"47d55d295b4ebb4cff5f732c8702943c303de12ed10e02dabc1abb6c70207d03"}
Jul 08 19:14:57.419579 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:57.419173    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:14:57.451644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:14:57.451587    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr" podStartSLOduration=0.797255946 podStartE2EDuration="5.451570311s" totalImagesPullingTime="4.654314365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:57.44876928 +0000 UTC m=+571.113331832" watchObservedRunningTime="2026-07-08 19:14:57.451570311 +0000 UTC m=+571.116132894"
Jul 08 19:15:03.454291 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:03.454260    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4tmjr"
Jul 08 19:15:11.052048 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.052011    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5cb95ddc78-psngm"]
Jul 08 19:15:11.061190 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.061158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.064963 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.064936    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q79gx\""
Jul 08 19:15:11.065120 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.065104    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 19:15:11.065980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.065966    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 19:15:11.072585 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.072562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cb95ddc78-psngm"]
Jul 08 19:15:11.093679 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.093637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.093876 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.093706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62zpq\" (UniqueName: \"kubernetes.io/projected/eec2464d-ca03-4e01-b8be-511f6c834035-kube-api-access-62zpq\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.194464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.194416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.194680 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.194488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62zpq\" (UniqueName: \"kubernetes.io/projected/eec2464d-ca03-4e01-b8be-511f6c834035-kube-api-access-62zpq\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.194680 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:11.194565    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 08 19:15:11.194680 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:11.194644    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert podName:eec2464d-ca03-4e01-b8be-511f6c834035 nodeName:}" failed. No retries permitted until 2026-07-08 19:15:11.69462711 +0000 UTC m=+585.359189645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert") pod "maas-controller-5cb95ddc78-psngm" (UID: "eec2464d-ca03-4e01-b8be-511f6c834035") : secret "maas-controller-webhook-cert" not found
Jul 08 19:15:11.211632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.211598    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62zpq\" (UniqueName: \"kubernetes.io/projected/eec2464d-ca03-4e01-b8be-511f6c834035-kube-api-access-62zpq\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.239929 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.239889    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-598545dc6b-dbt2f"]
Jul 08 19:15:11.242393 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.242371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.253785 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.253755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598545dc6b-dbt2f"]
Jul 08 19:15:11.295455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.295414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpgnf\" (UniqueName: \"kubernetes.io/projected/098dbb11-635b-4538-96f4-462286262b97-kube-api-access-qpgnf\") pod \"maas-controller-598545dc6b-dbt2f\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") " pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.295660 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.295483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/098dbb11-635b-4538-96f4-462286262b97-webhook-cert\") pod \"maas-controller-598545dc6b-dbt2f\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") " pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.396919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.396884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/098dbb11-635b-4538-96f4-462286262b97-webhook-cert\") pod \"maas-controller-598545dc6b-dbt2f\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") " pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.397131 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.396989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgnf\" (UniqueName: \"kubernetes.io/projected/098dbb11-635b-4538-96f4-462286262b97-kube-api-access-qpgnf\") pod \"maas-controller-598545dc6b-dbt2f\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") " pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.399607 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.399584    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/098dbb11-635b-4538-96f4-462286262b97-webhook-cert\") pod \"maas-controller-598545dc6b-dbt2f\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") " pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.407309 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.407273    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpgnf\" (UniqueName: \"kubernetes.io/projected/098dbb11-635b-4538-96f4-462286262b97-kube-api-access-qpgnf\") pod \"maas-controller-598545dc6b-dbt2f\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") " pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.553905 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.553864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:11.699775 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.699674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.702232 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.702209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598545dc6b-dbt2f"]
Jul 08 19:15:11.702751 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.702734    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert\") pod \"maas-controller-5cb95ddc78-psngm\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") " pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:11.704405 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:15:11.704382    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098dbb11_635b_4538_96f4_462286262b97.slice/crio-bcc3e1322eca565ad9a76249af5f5dd28c9a201acdf0e8aeb163876ffd83e655 WatchSource:0}: Error finding container bcc3e1322eca565ad9a76249af5f5dd28c9a201acdf0e8aeb163876ffd83e655: Status 404 returned error can't find the container with id bcc3e1322eca565ad9a76249af5f5dd28c9a201acdf0e8aeb163876ffd83e655
Jul 08 19:15:11.973333 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:11.973222    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:12.112142 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:12.112116    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cb95ddc78-psngm"]
Jul 08 19:15:12.114420 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:15:12.114391    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec2464d_ca03_4e01_b8be_511f6c834035.slice/crio-418977c6be0c6cc0e8fabe51fc7aa09ed0a0ff734e82e4bbffb55306bc2b475c WatchSource:0}: Error finding container 418977c6be0c6cc0e8fabe51fc7aa09ed0a0ff734e82e4bbffb55306bc2b475c: Status 404 returned error can't find the container with id 418977c6be0c6cc0e8fabe51fc7aa09ed0a0ff734e82e4bbffb55306bc2b475c
Jul 08 19:15:12.489448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:12.489402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cb95ddc78-psngm" event={"ID":"eec2464d-ca03-4e01-b8be-511f6c834035","Type":"ContainerStarted","Data":"418977c6be0c6cc0e8fabe51fc7aa09ed0a0ff734e82e4bbffb55306bc2b475c"}
Jul 08 19:15:12.490803 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:12.490766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598545dc6b-dbt2f" event={"ID":"098dbb11-635b-4538-96f4-462286262b97","Type":"ContainerStarted","Data":"bcc3e1322eca565ad9a76249af5f5dd28c9a201acdf0e8aeb163876ffd83e655"}
Jul 08 19:15:14.504380 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:14.504340    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cb95ddc78-psngm" event={"ID":"eec2464d-ca03-4e01-b8be-511f6c834035","Type":"ContainerStarted","Data":"02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6"}
Jul 08 19:15:14.504863 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:14.504459    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:14.506039 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:14.506017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598545dc6b-dbt2f" event={"ID":"098dbb11-635b-4538-96f4-462286262b97","Type":"ContainerStarted","Data":"50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de"}
Jul 08 19:15:14.506160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:14.506144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:14.528766 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:14.528711    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5cb95ddc78-psngm" podStartSLOduration=1.75575653 podStartE2EDuration="3.528695952s" totalImagesPullingTime="1.772939422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:14.527903475 +0000 UTC m=+588.192466027" watchObservedRunningTime="2026-07-08 19:15:14.528695952 +0000 UTC m=+588.193258502"
Jul 08 19:15:14.552993 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:14.552939    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-598545dc6b-dbt2f" podStartSLOduration=1.3796539970000001 podStartE2EDuration="3.552920944s" totalImagesPullingTime="2.173266947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:14.551278815 +0000 UTC m=+588.215841368" watchObservedRunningTime="2026-07-08 19:15:14.552920944 +0000 UTC m=+588.217483494"
Jul 08 19:15:25.516320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.516288    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:15:25.516721 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.516372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:25.589293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.589261    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cb95ddc78-psngm"]
Jul 08 19:15:25.589513 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.589488    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5cb95ddc78-psngm" podUID="eec2464d-ca03-4e01-b8be-511f6c834035" containerName="manager" containerID="cri-o://02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6" gracePeriod=30
Jul 08 19:15:25.835866 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.835841    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:25.939920 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.939884    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62zpq\" (UniqueName: \"kubernetes.io/projected/eec2464d-ca03-4e01-b8be-511f6c834035-kube-api-access-62zpq\") pod \"eec2464d-ca03-4e01-b8be-511f6c834035\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") "
Jul 08 19:15:25.940139 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.940005    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert\") pod \"eec2464d-ca03-4e01-b8be-511f6c834035\" (UID: \"eec2464d-ca03-4e01-b8be-511f6c834035\") "
Jul 08 19:15:25.942277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.942233    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eec2464d-ca03-4e01-b8be-511f6c834035-kube-api-access-62zpq" (OuterVolumeSpecName: "kube-api-access-62zpq") pod "eec2464d-ca03-4e01-b8be-511f6c834035" (UID: "eec2464d-ca03-4e01-b8be-511f6c834035"). InnerVolumeSpecName "kube-api-access-62zpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:15:25.942430 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:25.942310    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eec2464d-ca03-4e01-b8be-511f6c834035" (UID: "eec2464d-ca03-4e01-b8be-511f6c834035"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:15:26.041287 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.041246    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62zpq\" (UniqueName: \"kubernetes.io/projected/eec2464d-ca03-4e01-b8be-511f6c834035-kube-api-access-62zpq\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:15:26.041287 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.041276    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eec2464d-ca03-4e01-b8be-511f6c834035-webhook-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:15:26.569203 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.569161    2563 generic.go:356] "Generic (PLEG): container finished" podID="eec2464d-ca03-4e01-b8be-511f6c834035" containerID="02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6" exitCode=0
Jul 08 19:15:26.569203 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.569204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cb95ddc78-psngm" event={"ID":"eec2464d-ca03-4e01-b8be-511f6c834035","Type":"ContainerDied","Data":"02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6"}
Jul 08 19:15:26.569736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.569216    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cb95ddc78-psngm"
Jul 08 19:15:26.569736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.569253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cb95ddc78-psngm" event={"ID":"eec2464d-ca03-4e01-b8be-511f6c834035","Type":"ContainerDied","Data":"418977c6be0c6cc0e8fabe51fc7aa09ed0a0ff734e82e4bbffb55306bc2b475c"}
Jul 08 19:15:26.569736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.569279    2563 scope.go:117] "RemoveContainer" containerID="02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6"
Jul 08 19:15:26.579476 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.579455    2563 scope.go:117] "RemoveContainer" containerID="02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6"
Jul 08 19:15:26.579790 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:26.579769    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6\": container with ID starting with 02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6 not found: ID does not exist" containerID="02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6"
Jul 08 19:15:26.579858 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.579798    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6"} err="failed to get container status \"02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6\": rpc error: code = NotFound desc = could not find container \"02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6\": container with ID starting with 02c078fc350f94e59edd7eb76fe665994bdb879256c17d404c2414f9185d65b6 not found: ID does not exist"
Jul 08 19:15:26.614790 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.614752    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cb95ddc78-psngm"]
Jul 08 19:15:26.620648 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.620617    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5cb95ddc78-psngm"]
Jul 08 19:15:26.930799 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:26.930764    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eec2464d-ca03-4e01-b8be-511f6c834035" path="/var/lib/kubelet/pods/eec2464d-ca03-4e01-b8be-511f6c834035/volumes"
Jul 08 19:15:44.129825 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.129778    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"]
Jul 08 19:15:44.130457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.130387    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eec2464d-ca03-4e01-b8be-511f6c834035" containerName="manager"
Jul 08 19:15:44.130457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.130409    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec2464d-ca03-4e01-b8be-511f6c834035" containerName="manager"
Jul 08 19:15:44.130570 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.130493    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eec2464d-ca03-4e01-b8be-511f6c834035" containerName="manager"
Jul 08 19:15:44.147186 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.147156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.151618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.151516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 08 19:15:44.154964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.154634    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"]
Jul 08 19:15:44.201286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.201249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee048fc-ac58-4014-b916-12baa3d7a5e8-openshift-service-ca\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.201286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.201293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6n95\" (UniqueName: \"kubernetes.io/projected/6ee048fc-ac58-4014-b916-12baa3d7a5e8-kube-api-access-b6n95\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.201524 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.201373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.302252 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.302215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee048fc-ac58-4014-b916-12baa3d7a5e8-openshift-service-ca\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.302434 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.302264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6n95\" (UniqueName: \"kubernetes.io/projected/6ee048fc-ac58-4014-b916-12baa3d7a5e8-kube-api-access-b6n95\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.302434 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.302309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.302554 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:44.302453    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 08 19:15:44.302554 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:44.302520    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls podName:6ee048fc-ac58-4014-b916-12baa3d7a5e8 nodeName:}" failed. No retries permitted until 2026-07-08 19:15:44.802502107 +0000 UTC m=+618.467064638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls") pod "maas-api-6c45ffb8d8-klcgh" (UID: "6ee048fc-ac58-4014-b916-12baa3d7a5e8") : secret "maas-api-serving-cert" not found
Jul 08 19:15:44.302845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.302825    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee048fc-ac58-4014-b916-12baa3d7a5e8-openshift-service-ca\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.320918 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.320878    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6n95\" (UniqueName: \"kubernetes.io/projected/6ee048fc-ac58-4014-b916-12baa3d7a5e8-kube-api-access-b6n95\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.806913 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:44.806880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:44.807183 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:44.807013    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 08 19:15:44.807183 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:15:44.807078    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls podName:6ee048fc-ac58-4014-b916-12baa3d7a5e8 nodeName:}" failed. No retries permitted until 2026-07-08 19:15:45.807062405 +0000 UTC m=+619.471624934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls") pod "maas-api-6c45ffb8d8-klcgh" (UID: "6ee048fc-ac58-4014-b916-12baa3d7a5e8") : secret "maas-api-serving-cert" not found
Jul 08 19:15:45.003349 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.003312    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mbpvb"]
Jul 08 19:15:45.005972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.005952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.008917 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.008892    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vsw68\""
Jul 08 19:15:45.009051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.009012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40b6fb8-5369-4520-bb1c-edfbb53d245f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mbpvb\" (UID: \"e40b6fb8-5369-4520-bb1c-edfbb53d245f\") " pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.009186 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.009100    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 08 19:15:45.009186 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.009145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2w4\" (UniqueName: \"kubernetes.io/projected/e40b6fb8-5369-4520-bb1c-edfbb53d245f-kube-api-access-5t2w4\") pod \"payload-pre-processing-7964457696-mbpvb\" (UID: \"e40b6fb8-5369-4520-bb1c-edfbb53d245f\") " pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.024107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.021640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mbpvb"]
Jul 08 19:15:45.084246 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.084171    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-sk8l9"]
Jul 08 19:15:45.086682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.086657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.108556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.108522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-sk8l9"]
Jul 08 19:15:45.109805 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.109776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a0528a-79bf-4b59-b496-079fbe68da50-plugins-config-volume\") pod \"payload-processing-58f86c7798-sk8l9\" (UID: \"15a0528a-79bf-4b59-b496-079fbe68da50\") " pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.109933 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.109818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2w4\" (UniqueName: \"kubernetes.io/projected/e40b6fb8-5369-4520-bb1c-edfbb53d245f-kube-api-access-5t2w4\") pod \"payload-pre-processing-7964457696-mbpvb\" (UID: \"e40b6fb8-5369-4520-bb1c-edfbb53d245f\") " pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.109933 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.109844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62kcq\" (UniqueName: \"kubernetes.io/projected/15a0528a-79bf-4b59-b496-079fbe68da50-kube-api-access-62kcq\") pod \"payload-processing-58f86c7798-sk8l9\" (UID: \"15a0528a-79bf-4b59-b496-079fbe68da50\") " pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.110041 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.109997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40b6fb8-5369-4520-bb1c-edfbb53d245f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mbpvb\" (UID: \"e40b6fb8-5369-4520-bb1c-edfbb53d245f\") " pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.110914 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.110891    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40b6fb8-5369-4520-bb1c-edfbb53d245f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mbpvb\" (UID: \"e40b6fb8-5369-4520-bb1c-edfbb53d245f\") " pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.132166 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.132128    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2w4\" (UniqueName: \"kubernetes.io/projected/e40b6fb8-5369-4520-bb1c-edfbb53d245f-kube-api-access-5t2w4\") pod \"payload-pre-processing-7964457696-mbpvb\" (UID: \"e40b6fb8-5369-4520-bb1c-edfbb53d245f\") " pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.210626 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.210582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a0528a-79bf-4b59-b496-079fbe68da50-plugins-config-volume\") pod \"payload-processing-58f86c7798-sk8l9\" (UID: \"15a0528a-79bf-4b59-b496-079fbe68da50\") " pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.210780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.210634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62kcq\" (UniqueName: \"kubernetes.io/projected/15a0528a-79bf-4b59-b496-079fbe68da50-kube-api-access-62kcq\") pod \"payload-processing-58f86c7798-sk8l9\" (UID: \"15a0528a-79bf-4b59-b496-079fbe68da50\") " pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.211521 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.211495    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a0528a-79bf-4b59-b496-079fbe68da50-plugins-config-volume\") pod \"payload-processing-58f86c7798-sk8l9\" (UID: \"15a0528a-79bf-4b59-b496-079fbe68da50\") " pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.220313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.220283    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62kcq\" (UniqueName: \"kubernetes.io/projected/15a0528a-79bf-4b59-b496-079fbe68da50-kube-api-access-62kcq\") pod \"payload-processing-58f86c7798-sk8l9\" (UID: \"15a0528a-79bf-4b59-b496-079fbe68da50\") " pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.317429 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.317387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:45.398466 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.398428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:45.471617 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.471588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mbpvb"]
Jul 08 19:15:45.473291 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:15:45.473256    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40b6fb8_5369_4520_bb1c_edfbb53d245f.slice/crio-26037bda8bfa9961ef0aef360420b726955b3bfa0f1c83568111fd8889c85008 WatchSource:0}: Error finding container 26037bda8bfa9961ef0aef360420b726955b3bfa0f1c83568111fd8889c85008: Status 404 returned error can't find the container with id 26037bda8bfa9961ef0aef360420b726955b3bfa0f1c83568111fd8889c85008
Jul 08 19:15:45.595511 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.593960    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-sk8l9"]
Jul 08 19:15:45.604107 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:15:45.603201    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a0528a_79bf_4b59_b496_079fbe68da50.slice/crio-a287a91305ebf1f6dd9b5ddefddbabd22ef68801eaab808e8e343edf8919c454 WatchSource:0}: Error finding container a287a91305ebf1f6dd9b5ddefddbabd22ef68801eaab808e8e343edf8919c454: Status 404 returned error can't find the container with id a287a91305ebf1f6dd9b5ddefddbabd22ef68801eaab808e8e343edf8919c454
Jul 08 19:15:45.659350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.659248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb" event={"ID":"e40b6fb8-5369-4520-bb1c-edfbb53d245f","Type":"ContainerStarted","Data":"26037bda8bfa9961ef0aef360420b726955b3bfa0f1c83568111fd8889c85008"}
Jul 08 19:15:45.660360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.660338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9" event={"ID":"15a0528a-79bf-4b59-b496-079fbe68da50","Type":"ContainerStarted","Data":"a287a91305ebf1f6dd9b5ddefddbabd22ef68801eaab808e8e343edf8919c454"}
Jul 08 19:15:45.820386 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.820353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:45.823220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.823188    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee048fc-ac58-4014-b916-12baa3d7a5e8-maas-api-tls\") pod \"maas-api-6c45ffb8d8-klcgh\" (UID: \"6ee048fc-ac58-4014-b916-12baa3d7a5e8\") " pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:45.958814 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:45.958726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:46.198228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:46.197466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"]
Jul 08 19:15:46.206131 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:15:46.206002    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee048fc_ac58_4014_b916_12baa3d7a5e8.slice/crio-3a6855c3990f4ead71606e6ebb96081b956624868d8fef60783dd2aae9a86b31 WatchSource:0}: Error finding container 3a6855c3990f4ead71606e6ebb96081b956624868d8fef60783dd2aae9a86b31: Status 404 returned error can't find the container with id 3a6855c3990f4ead71606e6ebb96081b956624868d8fef60783dd2aae9a86b31
Jul 08 19:15:46.666180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:46.666139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh" event={"ID":"6ee048fc-ac58-4014-b916-12baa3d7a5e8","Type":"ContainerStarted","Data":"3a6855c3990f4ead71606e6ebb96081b956624868d8fef60783dd2aae9a86b31"}
Jul 08 19:15:47.672985 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:47.672942    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb" event={"ID":"e40b6fb8-5369-4520-bb1c-edfbb53d245f","Type":"ContainerStarted","Data":"7c8918c158881561ae53975dfbc63cecbb7dc3075e7eeb374fa2c75d9528628f"}
Jul 08 19:15:47.673508 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:47.673017    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:47.675118 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:47.675028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9" event={"ID":"15a0528a-79bf-4b59-b496-079fbe68da50","Type":"ContainerStarted","Data":"241a385a9d2daf0ff2dea634af19477c06ccbc37caccebf561791ebddd77144b"}
Jul 08 19:15:47.675349 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:47.675302    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:47.717026 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:47.716952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb" podStartSLOduration=1.8391574450000001 podStartE2EDuration="3.716926894s" totalImagesPullingTime="1.877769449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:47.696694359 +0000 UTC m=+621.361256914" watchObservedRunningTime="2026-07-08 19:15:47.716926894 +0000 UTC m=+621.381489438"
Jul 08 19:15:47.720152 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:47.720064    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9" podStartSLOduration=0.974813935 podStartE2EDuration="2.720042849s" totalImagesPullingTime="1.745228914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:47.719716881 +0000 UTC m=+621.384279432" watchObservedRunningTime="2026-07-08 19:15:47.720042849 +0000 UTC m=+621.384605402"
Jul 08 19:15:48.684737 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:48.684696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh" event={"ID":"6ee048fc-ac58-4014-b916-12baa3d7a5e8","Type":"ContainerStarted","Data":"94fa094a69d42cc4d3bd360c0dd919a3aa205d5b83bded4500db0c71dc2fdfbc"}
Jul 08 19:15:48.685249 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:48.685225    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:48.703455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:48.703397    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh" podStartSLOduration=2.584447561 podStartE2EDuration="4.703375373s" totalImagesPullingTime="2.118927812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:48.702158647 +0000 UTC m=+622.366721200" watchObservedRunningTime="2026-07-08 19:15:48.703375373 +0000 UTC m=+622.367937928"
Jul 08 19:15:52.707385 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:52.707345    2563 generic.go:356] "Generic (PLEG): container finished" podID="566a6cc7-9156-43fe-a1fa-853634067c32" containerID="d8ed61d4501145b50bc78517ba73bd5d123ab27a8556bff6603bd691a2569e81" exitCode=0
Jul 08 19:15:52.707798 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:52.707424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" event={"ID":"566a6cc7-9156-43fe-a1fa-853634067c32","Type":"ContainerDied","Data":"d8ed61d4501145b50bc78517ba73bd5d123ab27a8556bff6603bd691a2569e81"}
Jul 08 19:15:52.707847 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:52.707809    2563 scope.go:117] "RemoveContainer" containerID="d8ed61d4501145b50bc78517ba73bd5d123ab27a8556bff6603bd691a2569e81"
Jul 08 19:15:52.708812 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:52.708781    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:15:52.708912 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:52.708863    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:15:52.708973 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:52.708912    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:15:53.714036 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:53.713994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" event={"ID":"566a6cc7-9156-43fe-a1fa-853634067c32","Type":"ContainerStarted","Data":"08ea189b06734a62c98d5b2eb673a462c5bcb299200dc26a71176a98afd16a47"}
Jul 08 19:15:53.898881 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:53.898825    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:15:53.899217 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:53.899179    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9mvc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 08 19:15:53.899321 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:53.899252    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" podUID="566a6cc7-9156-43fe-a1fa-853634067c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 08 19:15:54.899159 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:54.899122    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9mvc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 08 19:15:54.899755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:54.899721    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" podUID="566a6cc7-9156-43fe-a1fa-853634067c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 08 19:15:55.703430 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:55.703397    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c45ffb8d8-klcgh"
Jul 08 19:15:55.898731 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:55.898690    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9mvc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 08 19:15:55.899033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:55.899002    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" podUID="566a6cc7-9156-43fe-a1fa-853634067c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 08 19:15:56.898966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:56.898925    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9mvc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 08 19:15:56.899993 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:56.899574    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" podUID="566a6cc7-9156-43fe-a1fa-853634067c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 08 19:15:57.899157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:57.899113    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9mvc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 08 19:15:57.899678 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:57.899186    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4" podUID="566a6cc7-9156-43fe-a1fa-853634067c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 08 19:15:58.686996 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:58.686957    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-sk8l9"
Jul 08 19:15:58.687236 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:58.687166    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-mbpvb"
Jul 08 19:15:58.902514 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:58.902480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:15:58.902934 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:58.902800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:15:58.903469 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:15:58.903450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9mvc4"
Jul 08 19:16:07.211794 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.211758    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"]
Jul 08 19:16:07.215030 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.215010    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.218833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.218798    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mc8ln\""
Jul 08 19:16:07.218998 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.218876    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 08 19:16:07.218998 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.218805    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 08 19:16:07.219152 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.219138    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 08 19:16:07.225135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.225104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"]
Jul 08 19:16:07.328871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.328831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbb9f849-3835-4632-9a70-a253b32e5b75-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.328871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.328878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.329127 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.328909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.329127 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.328975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.329127 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.328998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb74j\" (UniqueName: \"kubernetes.io/projected/fbb9f849-3835-4632-9a70-a253b32e5b75-kube-api-access-kb74j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.329127 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.329021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb74j\" (UniqueName: \"kubernetes.io/projected/fbb9f849-3835-4632-9a70-a253b32e5b75-kube-api-access-kb74j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbb9f849-3835-4632-9a70-a253b32e5b75-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430546    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430626 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430581    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.430665 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.430647    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.432725 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.432697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbb9f849-3835-4632-9a70-a253b32e5b75-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.432944 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.432927    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbb9f849-3835-4632-9a70-a253b32e5b75-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.438818 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.438795    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb74j\" (UniqueName: \"kubernetes.io/projected/fbb9f849-3835-4632-9a70-a253b32e5b75-kube-api-access-kb74j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs\" (UID: \"fbb9f849-3835-4632-9a70-a253b32e5b75\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.527043 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.526944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:07.672527 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.672499    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"]
Jul 08 19:16:07.674781 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:16:07.674754    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb9f849_3835_4632_9a70_a253b32e5b75.slice/crio-8b454abaf74306728f7130feee975967376032bc5b21553c32d389e9765cbcf6 WatchSource:0}: Error finding container 8b454abaf74306728f7130feee975967376032bc5b21553c32d389e9765cbcf6: Status 404 returned error can't find the container with id 8b454abaf74306728f7130feee975967376032bc5b21553c32d389e9765cbcf6
Jul 08 19:16:07.794531 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:07.794442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs" event={"ID":"fbb9f849-3835-4632-9a70-a253b32e5b75","Type":"ContainerStarted","Data":"8b454abaf74306728f7130feee975967376032bc5b21553c32d389e9765cbcf6"}
Jul 08 19:16:09.807037 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:09.806998    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs" event={"ID":"fbb9f849-3835-4632-9a70-a253b32e5b75","Type":"ContainerStarted","Data":"e73d31e26e9c1cff6f39edfdce48e795e1f8bb66feebd6c7574caedaecbdf910"}
Jul 08 19:16:09.828724 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:09.828668    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs" podStartSLOduration=1.694320862 podStartE2EDuration="2.828652255s" totalImagesPullingTime="1.134331393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:09.828129235 +0000 UTC m=+643.492691786" watchObservedRunningTime="2026-07-08 19:16:09.828652255 +0000 UTC m=+643.493214805"
Jul 08 19:16:17.527240 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:17.527198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:17.527717 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:17.527314    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:17.544261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:17.544223    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:17.856958 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:17.856932    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs"
Jul 08 19:16:26.698305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.698265    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"]
Jul 08 19:16:26.703807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.703784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.706726 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.706701    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 08 19:16:26.715258 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.715226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"]
Jul 08 19:16:26.814444 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.814411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e8eb85b-2676-4a71-baa8-981698af7a3d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.814444 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.814448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4fzk\" (UniqueName: \"kubernetes.io/projected/8e8eb85b-2676-4a71-baa8-981698af7a3d-kube-api-access-h4fzk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.814659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.814470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.814659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.814553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.814659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.814627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.814659 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.814659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e8eb85b-2676-4a71-baa8-981698af7a3d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4fzk\" (UniqueName: \"kubernetes.io/projected/8e8eb85b-2676-4a71-baa8-981698af7a3d-kube-api-access-h4fzk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915700 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915604    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.915826 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.915796    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.916171 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.916149    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.918140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.918120    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e8eb85b-2676-4a71-baa8-981698af7a3d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.918897 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.918881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 08 19:16:26.925502 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.925466    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4fzk\" (UniqueName: \"kubernetes.io/projected/8e8eb85b-2676-4a71-baa8-981698af7a3d-kube-api-access-h4fzk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:26.929035 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:26.929003    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e8eb85b-2676-4a71-baa8-981698af7a3d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7q8fd\" (UID: \"8e8eb85b-2676-4a71-baa8-981698af7a3d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:27.017741 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:27.017660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:27.169493 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:27.169470    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"]
Jul 08 19:16:27.170661 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:16:27.170633    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8eb85b_2676_4a71_baa8_981698af7a3d.slice/crio-c09dd74590d956d3243c07b44e60e41136372587a0b02da1b2d7cb03c048b06e WatchSource:0}: Error finding container c09dd74590d956d3243c07b44e60e41136372587a0b02da1b2d7cb03c048b06e: Status 404 returned error can't find the container with id c09dd74590d956d3243c07b44e60e41136372587a0b02da1b2d7cb03c048b06e
Jul 08 19:16:27.172157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:27.172139    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 19:16:27.901034 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:27.900992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd" event={"ID":"8e8eb85b-2676-4a71-baa8-981698af7a3d","Type":"ContainerStarted","Data":"46fcb0542a02e1bd59cc9418d6e0bd48ed5d670fa3261534b1e35dc4337b3957"}
Jul 08 19:16:27.901464 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:27.901116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd" event={"ID":"8e8eb85b-2676-4a71-baa8-981698af7a3d","Type":"ContainerStarted","Data":"c09dd74590d956d3243c07b44e60e41136372587a0b02da1b2d7cb03c048b06e"}
Jul 08 19:16:27.923053 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:27.923000    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd" podStartSLOduration=1.7334926670000002 podStartE2EDuration="1.922982699s" totalImagesPullingTime="189.490032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:27.9216114 +0000 UTC m=+661.586173951" watchObservedRunningTime="2026-07-08 19:16:27.922982699 +0000 UTC m=+661.587545250"
Jul 08 19:16:37.018677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:37.018632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:37.019077 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:37.018706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:37.031273 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:37.031248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:37.959892 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:37.959862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7q8fd"
Jul 08 19:16:56.242805 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.242770    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"]
Jul 08 19:16:56.245779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.245757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.257500 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.257475    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 08 19:16:56.269143 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.269110    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"]
Jul 08 19:16:56.305954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.305911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7acdca82-402e-4340-8c44-53aff3891538-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.306166 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.306016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.306166 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.306055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.306166 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.306161    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtgqw\" (UniqueName: \"kubernetes.io/projected/7acdca82-402e-4340-8c44-53aff3891538-kube-api-access-wtgqw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.306277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.306192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.306277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.306213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.406943 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.406907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtgqw\" (UniqueName: \"kubernetes.io/projected/7acdca82-402e-4340-8c44-53aff3891538-kube-api-access-wtgqw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7acdca82-402e-4340-8c44-53aff3891538-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407441 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407394    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407453    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.407556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.407516    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.409607 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.409582    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7acdca82-402e-4340-8c44-53aff3891538-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.409743 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.409712    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7acdca82-402e-4340-8c44-53aff3891538-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.416602 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.416575    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtgqw\" (UniqueName: \"kubernetes.io/projected/7acdca82-402e-4340-8c44-53aff3891538-kube-api-access-wtgqw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7\" (UID: \"7acdca82-402e-4340-8c44-53aff3891538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.557278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.557193    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:16:56.706679 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:56.706654    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"]
Jul 08 19:16:56.708472 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:16:56.708443    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7acdca82_402e_4340_8c44_53aff3891538.slice/crio-d4c0a58a5f1ea42cdc077fd22aeac21fe071abe40329eb34cce1a50a260c2352 WatchSource:0}: Error finding container d4c0a58a5f1ea42cdc077fd22aeac21fe071abe40329eb34cce1a50a260c2352: Status 404 returned error can't find the container with id d4c0a58a5f1ea42cdc077fd22aeac21fe071abe40329eb34cce1a50a260c2352
Jul 08 19:16:57.045377 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:57.045342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7" event={"ID":"7acdca82-402e-4340-8c44-53aff3891538","Type":"ContainerStarted","Data":"191c3c6ee059fdc3c917eaaa2dbc4cb71f154ebbe88f3eea885bba9ffe0b6001"}
Jul 08 19:16:57.045607 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:57.045395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7" event={"ID":"7acdca82-402e-4340-8c44-53aff3891538","Type":"ContainerStarted","Data":"d4c0a58a5f1ea42cdc077fd22aeac21fe071abe40329eb34cce1a50a260c2352"}
Jul 08 19:16:57.063966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:16:57.063903    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7" podStartSLOduration=0.858398329 podStartE2EDuration="1.063885504s" totalImagesPullingTime="205.487175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:57.063122669 +0000 UTC m=+690.727685222" watchObservedRunningTime="2026-07-08 19:16:57.063885504 +0000 UTC m=+690.728448056"
Jul 08 19:17:06.558155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:17:06.558078    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:17:06.558679 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:17:06.558215    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:17:06.570957 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:17:06.570927    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:17:07.108655 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:17:07.108625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7"
Jul 08 19:20:05.428107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.428046    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-598545dc6b-dbt2f"]
Jul 08 19:20:05.428746 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.428356    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-598545dc6b-dbt2f" podUID="098dbb11-635b-4538-96f4-462286262b97" containerName="manager" containerID="cri-o://50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de" gracePeriod=30
Jul 08 19:20:05.511727 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.511678    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-598545dc6b-dbt2f" podUID="098dbb11-635b-4538-96f4-462286262b97" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.36:8081/readyz\": dial tcp 10.134.0.36:8081: connect: connection refused"
Jul 08 19:20:05.678635 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.678556    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:20:05.759350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.759314    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpgnf\" (UniqueName: \"kubernetes.io/projected/098dbb11-635b-4538-96f4-462286262b97-kube-api-access-qpgnf\") pod \"098dbb11-635b-4538-96f4-462286262b97\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") "
Jul 08 19:20:05.759540 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.759398    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/098dbb11-635b-4538-96f4-462286262b97-webhook-cert\") pod \"098dbb11-635b-4538-96f4-462286262b97\" (UID: \"098dbb11-635b-4538-96f4-462286262b97\") "
Jul 08 19:20:05.761449 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.761423    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098dbb11-635b-4538-96f4-462286262b97-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "098dbb11-635b-4538-96f4-462286262b97" (UID: "098dbb11-635b-4538-96f4-462286262b97"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:20:05.761563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.761484    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098dbb11-635b-4538-96f4-462286262b97-kube-api-access-qpgnf" (OuterVolumeSpecName: "kube-api-access-qpgnf") pod "098dbb11-635b-4538-96f4-462286262b97" (UID: "098dbb11-635b-4538-96f4-462286262b97"). InnerVolumeSpecName "kube-api-access-qpgnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:20:05.860633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.860597    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/098dbb11-635b-4538-96f4-462286262b97-webhook-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:20:05.860633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.860628    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpgnf\" (UniqueName: \"kubernetes.io/projected/098dbb11-635b-4538-96f4-462286262b97-kube-api-access-qpgnf\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:20:05.992260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.992166    2563 generic.go:356] "Generic (PLEG): container finished" podID="098dbb11-635b-4538-96f4-462286262b97" containerID="50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de" exitCode=0
Jul 08 19:20:05.992260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.992213    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598545dc6b-dbt2f"
Jul 08 19:20:05.992260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.992250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598545dc6b-dbt2f" event={"ID":"098dbb11-635b-4538-96f4-462286262b97","Type":"ContainerDied","Data":"50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de"}
Jul 08 19:20:05.992542 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.992304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598545dc6b-dbt2f" event={"ID":"098dbb11-635b-4538-96f4-462286262b97","Type":"ContainerDied","Data":"bcc3e1322eca565ad9a76249af5f5dd28c9a201acdf0e8aeb163876ffd83e655"}
Jul 08 19:20:05.992542 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:05.992326    2563 scope.go:117] "RemoveContainer" containerID="50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de"
Jul 08 19:20:06.001360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.001344    2563 scope.go:117] "RemoveContainer" containerID="50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de"
Jul 08 19:20:06.001640 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:20:06.001620    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de\": container with ID starting with 50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de not found: ID does not exist" containerID="50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de"
Jul 08 19:20:06.001690 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.001650    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de"} err="failed to get container status \"50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de\": rpc error: code = NotFound desc = could not find container \"50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de\": container with ID starting with 50b0f9cc105bee68329b10900f29576de44566699bb70a200673bc86d39506de not found: ID does not exist"
Jul 08 19:20:06.016284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.016253    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-598545dc6b-dbt2f"]
Jul 08 19:20:06.022444 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.022418    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-598545dc6b-dbt2f"]
Jul 08 19:20:06.896484 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.896445    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-598545dc6b-ncr96"]
Jul 08 19:20:06.897231 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.897206    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="098dbb11-635b-4538-96f4-462286262b97" containerName="manager"
Jul 08 19:20:06.897231 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.897231    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="098dbb11-635b-4538-96f4-462286262b97" containerName="manager"
Jul 08 19:20:06.897436 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.897350    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="098dbb11-635b-4538-96f4-462286262b97" containerName="manager"
Jul 08 19:20:06.901728 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.901703    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:06.905656 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.905629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q79gx\""
Jul 08 19:20:06.905799 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.905673    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 19:20:06.905799 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.905635    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 19:20:06.911441 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.911408    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598545dc6b-ncr96"]
Jul 08 19:20:06.930313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.930278    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="098dbb11-635b-4538-96f4-462286262b97" path="/var/lib/kubelet/pods/098dbb11-635b-4538-96f4-462286262b97/volumes"
Jul 08 19:20:06.972000 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.971955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmsjz\" (UniqueName: \"kubernetes.io/projected/9f4ae60b-2c83-40c5-a2c9-4fb477d05a00-kube-api-access-lmsjz\") pod \"maas-controller-598545dc6b-ncr96\" (UID: \"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00\") " pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:06.972223 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:06.972024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f4ae60b-2c83-40c5-a2c9-4fb477d05a00-webhook-cert\") pod \"maas-controller-598545dc6b-ncr96\" (UID: \"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00\") " pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:07.073038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:07.072990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmsjz\" (UniqueName: \"kubernetes.io/projected/9f4ae60b-2c83-40c5-a2c9-4fb477d05a00-kube-api-access-lmsjz\") pod \"maas-controller-598545dc6b-ncr96\" (UID: \"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00\") " pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:07.073276 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:07.073116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f4ae60b-2c83-40c5-a2c9-4fb477d05a00-webhook-cert\") pod \"maas-controller-598545dc6b-ncr96\" (UID: \"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00\") " pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:07.075838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:07.075800    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f4ae60b-2c83-40c5-a2c9-4fb477d05a00-webhook-cert\") pod \"maas-controller-598545dc6b-ncr96\" (UID: \"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00\") " pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:07.082643 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:07.082606    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmsjz\" (UniqueName: \"kubernetes.io/projected/9f4ae60b-2c83-40c5-a2c9-4fb477d05a00-kube-api-access-lmsjz\") pod \"maas-controller-598545dc6b-ncr96\" (UID: \"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00\") " pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:07.214105 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:07.213982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:07.357657 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:07.357626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598545dc6b-ncr96"]
Jul 08 19:20:07.359204 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:20:07.359178    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4ae60b_2c83_40c5_a2c9_4fb477d05a00.slice/crio-0ce908c32ad529d452632b12485d589f08b2be61c877b4e6fbfbacb0307d4d5e WatchSource:0}: Error finding container 0ce908c32ad529d452632b12485d589f08b2be61c877b4e6fbfbacb0307d4d5e: Status 404 returned error can't find the container with id 0ce908c32ad529d452632b12485d589f08b2be61c877b4e6fbfbacb0307d4d5e
Jul 08 19:20:08.005645 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:08.005609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598545dc6b-ncr96" event={"ID":"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00","Type":"ContainerStarted","Data":"84b493f9856963468bd644e5cd5e11fafb5b50bf8079225f9306b514adac0598"}
Jul 08 19:20:08.006157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:08.005664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598545dc6b-ncr96" event={"ID":"9f4ae60b-2c83-40c5-a2c9-4fb477d05a00","Type":"ContainerStarted","Data":"0ce908c32ad529d452632b12485d589f08b2be61c877b4e6fbfbacb0307d4d5e"}
Jul 08 19:20:08.006157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:08.005696    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:20:08.028916 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:08.028852    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-598545dc6b-ncr96" podStartSLOduration=1.527852915 podStartE2EDuration="2.028837138s" totalImagesPullingTime="500.984223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:20:08.026527491 +0000 UTC m=+881.691090041" watchObservedRunningTime="2026-07-08 19:20:08.028837138 +0000 UTC m=+881.693399690"
Jul 08 19:20:19.017133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:20:19.017025    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-598545dc6b-ncr96"
Jul 08 19:25:00.227654 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:00.227566    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"]
Jul 08 19:25:00.228180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:00.227771    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r" podUID="a5a88e71-6b53-4ee0-9753-5c762aa4cb45" containerName="manager" containerID="cri-o://11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6" gracePeriod=10
Jul 08 19:25:02.885333 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:02.885296    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:25:02.947898 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:02.947802    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-extensions-socket-volume\") pod \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") "
Jul 08 19:25:02.948070 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:02.947931    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kclnt\" (UniqueName: \"kubernetes.io/projected/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-kube-api-access-kclnt\") pod \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\" (UID: \"a5a88e71-6b53-4ee0-9753-5c762aa4cb45\") "
Jul 08 19:25:02.948206 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:02.948183    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a5a88e71-6b53-4ee0-9753-5c762aa4cb45" (UID: "a5a88e71-6b53-4ee0-9753-5c762aa4cb45"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:25:02.950174 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:02.950148    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-kube-api-access-kclnt" (OuterVolumeSpecName: "kube-api-access-kclnt") pod "a5a88e71-6b53-4ee0-9753-5c762aa4cb45" (UID: "a5a88e71-6b53-4ee0-9753-5c762aa4cb45"). InnerVolumeSpecName "kube-api-access-kclnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:25:03.049051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.049007    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kclnt\" (UniqueName: \"kubernetes.io/projected/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-kube-api-access-kclnt\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:25:03.049051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.049040    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5a88e71-6b53-4ee0-9753-5c762aa4cb45-extensions-socket-volume\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:25:03.517360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.517324    2563 generic.go:356] "Generic (PLEG): container finished" podID="a5a88e71-6b53-4ee0-9753-5c762aa4cb45" containerID="11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6" exitCode=0
Jul 08 19:25:03.517552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.517372    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"
Jul 08 19:25:03.517552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.517413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r" event={"ID":"a5a88e71-6b53-4ee0-9753-5c762aa4cb45","Type":"ContainerDied","Data":"11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6"}
Jul 08 19:25:03.517552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.517478    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r" event={"ID":"a5a88e71-6b53-4ee0-9753-5c762aa4cb45","Type":"ContainerDied","Data":"c123dfdd74a3cea8347570b1ef6eea61619891a46b9d182bc25e31697cfda142"}
Jul 08 19:25:03.517552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.517502    2563 scope.go:117] "RemoveContainer" containerID="11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6"
Jul 08 19:25:03.527900 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.527877    2563 scope.go:117] "RemoveContainer" containerID="11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6"
Jul 08 19:25:03.528269 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:25:03.528250    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6\": container with ID starting with 11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6 not found: ID does not exist" containerID="11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6"
Jul 08 19:25:03.528362 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.528277    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6"} err="failed to get container status \"11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6\": rpc error: code = NotFound desc = could not find container \"11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6\": container with ID starting with 11284811ce1ba90e80d2496ab292a8573aeff18395a235d79dba193d3d9b3ae6 not found: ID does not exist"
Jul 08 19:25:03.541287 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.541258    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"]
Jul 08 19:25:03.549820 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:03.549782    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqb7r"]
Jul 08 19:25:04.930862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:25:04.930822    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5a88e71-6b53-4ee0-9753-5c762aa4cb45" path="/var/lib/kubelet/pods/a5a88e71-6b53-4ee0-9753-5c762aa4cb45/volumes"
Jul 08 19:26:01.435139 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.435099    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"]
Jul 08 19:26:01.435685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.435520    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5a88e71-6b53-4ee0-9753-5c762aa4cb45" containerName="manager"
Jul 08 19:26:01.435685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.435532    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5a88e71-6b53-4ee0-9753-5c762aa4cb45" containerName="manager"
Jul 08 19:26:01.435685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.435598    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5a88e71-6b53-4ee0-9753-5c762aa4cb45" containerName="manager"
Jul 08 19:26:01.438871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.438851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.441718 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.441688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qjbt8\""
Jul 08 19:26:01.451509 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.451474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"]
Jul 08 19:26:01.516147 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.516068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng2l8\" (UniqueName: \"kubernetes.io/projected/039f8156-8b28-45ef-8b3f-8ca26ec8417a-kube-api-access-ng2l8\") pod \"kuadrant-operator-controller-manager-86878c769f-c6p9g\" (UID: \"039f8156-8b28-45ef-8b3f-8ca26ec8417a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.516336 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.516268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/039f8156-8b28-45ef-8b3f-8ca26ec8417a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c6p9g\" (UID: \"039f8156-8b28-45ef-8b3f-8ca26ec8417a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.616782 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.616733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/039f8156-8b28-45ef-8b3f-8ca26ec8417a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c6p9g\" (UID: \"039f8156-8b28-45ef-8b3f-8ca26ec8417a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.616905 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.616820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng2l8\" (UniqueName: \"kubernetes.io/projected/039f8156-8b28-45ef-8b3f-8ca26ec8417a-kube-api-access-ng2l8\") pod \"kuadrant-operator-controller-manager-86878c769f-c6p9g\" (UID: \"039f8156-8b28-45ef-8b3f-8ca26ec8417a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.617191 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.617173    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/039f8156-8b28-45ef-8b3f-8ca26ec8417a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c6p9g\" (UID: \"039f8156-8b28-45ef-8b3f-8ca26ec8417a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.626933 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.626901    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng2l8\" (UniqueName: \"kubernetes.io/projected/039f8156-8b28-45ef-8b3f-8ca26ec8417a-kube-api-access-ng2l8\") pod \"kuadrant-operator-controller-manager-86878c769f-c6p9g\" (UID: \"039f8156-8b28-45ef-8b3f-8ca26ec8417a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.750345 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.750252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:01.910482 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:01.910452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"]
Jul 08 19:26:01.912851 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:26:01.912823    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039f8156_8b28_45ef_8b3f_8ca26ec8417a.slice/crio-4e969d18ffd170a6bb83dcc1e46c3ec26670e065fa4eccb39f4d01355bb91045 WatchSource:0}: Error finding container 4e969d18ffd170a6bb83dcc1e46c3ec26670e065fa4eccb39f4d01355bb91045: Status 404 returned error can't find the container with id 4e969d18ffd170a6bb83dcc1e46c3ec26670e065fa4eccb39f4d01355bb91045
Jul 08 19:26:02.813734 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:02.813694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g" event={"ID":"039f8156-8b28-45ef-8b3f-8ca26ec8417a","Type":"ContainerStarted","Data":"5d1f4a13439f659287aa78467c384c3adbc5f484900b30bd56a98afaf1304e8d"}
Jul 08 19:26:02.814222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:02.813755    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g" event={"ID":"039f8156-8b28-45ef-8b3f-8ca26ec8417a","Type":"ContainerStarted","Data":"4e969d18ffd170a6bb83dcc1e46c3ec26670e065fa4eccb39f4d01355bb91045"}
Jul 08 19:26:02.814222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:02.813783    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:26:02.838400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:02.838342    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g" podStartSLOduration=1.8383233300000001 podStartE2EDuration="1.83832333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:26:02.835256867 +0000 UTC m=+1236.499819430" watchObservedRunningTime="2026-07-08 19:26:02.83832333 +0000 UTC m=+1236.502885881"
Jul 08 19:26:13.821390 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:26:13.821358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6p9g"
Jul 08 19:30:00.153588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:00.153543    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"]
Jul 08 19:30:00.157378 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:00.157354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"
Jul 08 19:30:00.166505 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:00.166484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"
Jul 08 19:30:00.178363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:00.178335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"]
Jul 08 19:30:00.308677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:00.308653    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"]
Jul 08 19:30:00.310702 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:30:00.310679    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2878e5c8_d0d4_4fd5_8739_d4001a4f8efd.slice/crio-63ef895120cfd33bb483e6c8e350447874629722cf21bb037fe9d5878e54f9d8 WatchSource:0}: Error finding container 63ef895120cfd33bb483e6c8e350447874629722cf21bb037fe9d5878e54f9d8: Status 404 returned error can't find the container with id 63ef895120cfd33bb483e6c8e350447874629722cf21bb037fe9d5878e54f9d8
Jul 08 19:30:00.313058 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:00.313039    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 19:30:01.008239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:01.008196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r" event={"ID":"2878e5c8-d0d4-4fd5-8739-d4001a4f8efd","Type":"ContainerStarted","Data":"63ef895120cfd33bb483e6c8e350447874629722cf21bb037fe9d5878e54f9d8"}
Jul 08 19:30:03.020552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:03.020512    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r" event={"ID":"2878e5c8-d0d4-4fd5-8739-d4001a4f8efd","Type":"ContainerStarted","Data":"05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd"}
Jul 08 19:30:03.044691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:30:03.044578    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r" podStartSLOduration=0.882966987 podStartE2EDuration="3.044559958s" totalImagesPullingTime="2.161592971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:30:03.043070853 +0000 UTC m=+1476.707633406" watchObservedRunningTime="2026-07-08 19:30:03.044559958 +0000 UTC m=+1476.709122507"
Jul 08 19:31:00.011707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:00.011618    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"]
Jul 08 19:31:00.012282 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:00.011847    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r" podUID="2878e5c8-d0d4-4fd5-8739-d4001a4f8efd" containerName="cleanup" containerID="cri-o://05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd" gracePeriod=30
Jul 08 19:31:30.223904 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.223868    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"
Jul 08 19:31:30.464852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.464814    2563 generic.go:356] "Generic (PLEG): container finished" podID="2878e5c8-d0d4-4fd5-8739-d4001a4f8efd" containerID="05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd" exitCode=137
Jul 08 19:31:30.465067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.464857    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"
Jul 08 19:31:30.465067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.464867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r" event={"ID":"2878e5c8-d0d4-4fd5-8739-d4001a4f8efd","Type":"ContainerDied","Data":"05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd"}
Jul 08 19:31:30.465067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.464920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r" event={"ID":"2878e5c8-d0d4-4fd5-8739-d4001a4f8efd","Type":"ContainerDied","Data":"63ef895120cfd33bb483e6c8e350447874629722cf21bb037fe9d5878e54f9d8"}
Jul 08 19:31:30.465067 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.464948    2563 scope.go:117] "RemoveContainer" containerID="05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd"
Jul 08 19:31:30.474109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.474044    2563 scope.go:117] "RemoveContainer" containerID="05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd"
Jul 08 19:31:30.474422 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:31:30.474406    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd\": container with ID starting with 05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd not found: ID does not exist" containerID="05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd"
Jul 08 19:31:30.474477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.474433    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd"} err="failed to get container status \"05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd\": rpc error: code = NotFound desc = could not find container \"05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd\": container with ID starting with 05a52a59a8229bdd511960bea990394be5ab48af2a9f3cf79b9c22992fd961bd not found: ID does not exist"
Jul 08 19:31:30.490527 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.490473    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"]
Jul 08 19:31:30.492099 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.492050    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725650-s5w2r"]
Jul 08 19:31:30.931700 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:31:30.931659    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2878e5c8-d0d4-4fd5-8739-d4001a4f8efd" path="/var/lib/kubelet/pods/2878e5c8-d0d4-4fd5-8739-d4001a4f8efd/volumes"
Jul 08 19:32:30.969860 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.969818    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"]
Jul 08 19:32:30.970494 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.970470    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2878e5c8-d0d4-4fd5-8739-d4001a4f8efd" containerName="cleanup"
Jul 08 19:32:30.970539 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.970499    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2878e5c8-d0d4-4fd5-8739-d4001a4f8efd" containerName="cleanup"
Jul 08 19:32:30.970634 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.970622    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2878e5c8-d0d4-4fd5-8739-d4001a4f8efd" containerName="cleanup"
Jul 08 19:32:30.975276 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.975255    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.978145 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.978118    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7b30e0e2-openshift-default-dockercfg-d482x\""
Jul 08 19:32:30.981667 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-certs\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.981800 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlqm4\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-kube-api-access-wlqm4\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.981800 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-data\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.981800 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-envoy\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.982062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-socket\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.982062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3954dd64-692c-4498-bccb-378411c65fa8-istiod-ca-cert\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.982062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.981999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-credential-socket\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.982275 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.982162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-istio-token\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.982275 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.982208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3954dd64-692c-4498-bccb-378411c65fa8-istio-podinfo\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:30.986956 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:30.986930    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"]
Jul 08 19:32:31.082993 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.082955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-envoy\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.082993 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.082993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-socket\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3954dd64-692c-4498-bccb-378411c65fa8-istiod-ca-cert\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-credential-socket\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-istio-token\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3954dd64-692c-4498-bccb-378411c65fa8-istio-podinfo\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-certs\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083537 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlqm4\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-kube-api-access-wlqm4\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.083537 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.083328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-data\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.084113 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.084045    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-socket\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.084238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.084145    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-credential-socket\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.084238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.084174    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-certs\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.084392 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.084371    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-data\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.084533 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.084513    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3954dd64-692c-4498-bccb-378411c65fa8-istiod-ca-cert\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.086573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.086538    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3954dd64-692c-4498-bccb-378411c65fa8-istio-podinfo\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.086735 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.086713    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-envoy\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.095522 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.095489    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-istio-token\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.096352 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.096289    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlqm4\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-kube-api-access-wlqm4\") pod \"e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") " pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.288404 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.288307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:31.439398 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.439367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"]
Jul 08 19:32:31.441734 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:32:31.441709    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3954dd64_692c_4498_bccb_378411c65fa8.slice/crio-d289d4bfca306200cc94c266700867720c051767a65af40b6674c5912ca8d339 WatchSource:0}: Error finding container d289d4bfca306200cc94c266700867720c051767a65af40b6674c5912ca8d339: Status 404 returned error can't find the container with id d289d4bfca306200cc94c266700867720c051767a65af40b6674c5912ca8d339
Jul 08 19:32:31.444255 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.444224    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:32:31.444358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.444305    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:32:31.444358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.444350    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:32:31.788669 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.788623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk" event={"ID":"3954dd64-692c-4498-bccb-378411c65fa8","Type":"ContainerStarted","Data":"acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a"}
Jul 08 19:32:31.788907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.788699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk" event={"ID":"3954dd64-692c-4498-bccb-378411c65fa8","Type":"ContainerStarted","Data":"d289d4bfca306200cc94c266700867720c051767a65af40b6674c5912ca8d339"}
Jul 08 19:32:31.811055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:31.810998    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk" podStartSLOduration=1.810978196 podStartE2EDuration="1.810978196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:31.809063516 +0000 UTC m=+1625.473626104" watchObservedRunningTime="2026-07-08 19:32:31.810978196 +0000 UTC m=+1625.475540746"
Jul 08 19:32:32.235749 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.235709    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"]
Jul 08 19:32:32.239505 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.239473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.242224 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.242196    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7b30e0e2-serving-cert\""
Jul 08 19:32:32.242385 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.242245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:32:32.254000 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.253964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"]
Jul 08 19:32:32.289061 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.289027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:32.293183 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.293140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.293337 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.293265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxwn2\" (UniqueName: \"kubernetes.io/projected/f995d345-1266-452d-968e-a6aa70ed215a-kube-api-access-sxwn2\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.293553 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.293366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995d345-1266-452d-968e-a6aa70ed215a-openshift-service-ca\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.294242 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.294220    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:32.393948 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.393908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxwn2\" (UniqueName: \"kubernetes.io/projected/f995d345-1266-452d-968e-a6aa70ed215a-kube-api-access-sxwn2\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.394193 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.393962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995d345-1266-452d-968e-a6aa70ed215a-openshift-service-ca\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.394193 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.394012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.394193 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:32.394143    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-serving-cert: secret "maas-api-e2e-ait-7b30e0e2-serving-cert" not found
Jul 08 19:32:32.394395 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:32.394211    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls podName:f995d345-1266-452d-968e-a6aa70ed215a nodeName:}" failed. No retries permitted until 2026-07-08 19:32:32.894191571 +0000 UTC m=+1626.558754099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls") pod "maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" (UID: "f995d345-1266-452d-968e-a6aa70ed215a") : secret "maas-api-e2e-ait-7b30e0e2-serving-cert" not found
Jul 08 19:32:32.394792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.394770    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995d345-1266-452d-968e-a6aa70ed215a-openshift-service-ca\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.403695 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.403661    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxwn2\" (UniqueName: \"kubernetes.io/projected/f995d345-1266-452d-968e-a6aa70ed215a-kube-api-access-sxwn2\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.794542 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.794511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:32.795682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.795662    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:32.899732 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.899691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:32.902374 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:32.902352    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls\") pod \"maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:33.152238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:33.152195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:33.302195 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:33.302158    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"]
Jul 08 19:32:33.305621 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:32:33.305585    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf995d345_1266_452d_968e_a6aa70ed215a.slice/crio-1ebdb1c98407a6c472c25859ddde55fb667a3865f4329c3f84560c00882dac85 WatchSource:0}: Error finding container 1ebdb1c98407a6c472c25859ddde55fb667a3865f4329c3f84560c00882dac85: Status 404 returned error can't find the container with id 1ebdb1c98407a6c472c25859ddde55fb667a3865f4329c3f84560c00882dac85
Jul 08 19:32:33.801151 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:33.801118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" event={"ID":"f995d345-1266-452d-968e-a6aa70ed215a","Type":"ContainerStarted","Data":"1ebdb1c98407a6c472c25859ddde55fb667a3865f4329c3f84560c00882dac85"}
Jul 08 19:32:34.809148 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:34.809041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" event={"ID":"f995d345-1266-452d-968e-a6aa70ed215a","Type":"ContainerStarted","Data":"05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2"}
Jul 08 19:32:34.809543 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:34.809281    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:34.830114 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:34.830043    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" podStartSLOduration=2.377429117 podStartE2EDuration="2.830024433s" totalImagesPullingTime="452.595316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:34.829314323 +0000 UTC m=+1628.493876874" watchObservedRunningTime="2026-07-08 19:32:34.830024433 +0000 UTC m=+1628.494586983"
Jul 08 19:32:37.465403 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:37.465365    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"]
Jul 08 19:32:37.465883 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:37.465570    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk" podUID="3954dd64-692c-4498-bccb-378411c65fa8" containerName="istio-proxy" containerID="cri-o://acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a" gracePeriod=30
Jul 08 19:32:37.893116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:37.888889    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"]
Jul 08 19:32:37.893116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:37.889164    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" podUID="f995d345-1266-452d-968e-a6aa70ed215a" containerName="maas-api" containerID="cri-o://05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2" gracePeriod=30
Jul 08 19:32:38.210392 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.210363    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:38.257100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.257039    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995d345-1266-452d-968e-a6aa70ed215a-openshift-service-ca\") pod \"f995d345-1266-452d-968e-a6aa70ed215a\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") "
Jul 08 19:32:38.257306 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.257131    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls\") pod \"f995d345-1266-452d-968e-a6aa70ed215a\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") "
Jul 08 19:32:38.257306 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.257195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxwn2\" (UniqueName: \"kubernetes.io/projected/f995d345-1266-452d-968e-a6aa70ed215a-kube-api-access-sxwn2\") pod \"f995d345-1266-452d-968e-a6aa70ed215a\" (UID: \"f995d345-1266-452d-968e-a6aa70ed215a\") "
Jul 08 19:32:38.257509 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.257485    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f995d345-1266-452d-968e-a6aa70ed215a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f995d345-1266-452d-968e-a6aa70ed215a" (UID: "f995d345-1266-452d-968e-a6aa70ed215a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:32:38.257625 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.257608    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995d345-1266-452d-968e-a6aa70ed215a-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:38.259693 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.259663    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f995d345-1266-452d-968e-a6aa70ed215a-kube-api-access-sxwn2" (OuterVolumeSpecName: "kube-api-access-sxwn2") pod "f995d345-1266-452d-968e-a6aa70ed215a" (UID: "f995d345-1266-452d-968e-a6aa70ed215a"). InnerVolumeSpecName "kube-api-access-sxwn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:32:38.259774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.259675    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f995d345-1266-452d-968e-a6aa70ed215a" (UID: "f995d345-1266-452d-968e-a6aa70ed215a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:32:38.358408 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.358369    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxwn2\" (UniqueName: \"kubernetes.io/projected/f995d345-1266-452d-968e-a6aa70ed215a-kube-api-access-sxwn2\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:38.358408 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.358404    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995d345-1266-452d-968e-a6aa70ed215a-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:38.830955 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.830917    2563 generic.go:356] "Generic (PLEG): container finished" podID="f995d345-1266-452d-968e-a6aa70ed215a" containerID="05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2" exitCode=0
Jul 08 19:32:38.831457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.830962    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"
Jul 08 19:32:38.831457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.831004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" event={"ID":"f995d345-1266-452d-968e-a6aa70ed215a","Type":"ContainerDied","Data":"05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2"}
Jul 08 19:32:38.831457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.831054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677" event={"ID":"f995d345-1266-452d-968e-a6aa70ed215a","Type":"ContainerDied","Data":"1ebdb1c98407a6c472c25859ddde55fb667a3865f4329c3f84560c00882dac85"}
Jul 08 19:32:38.831457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.831077    2563 scope.go:117] "RemoveContainer" containerID="05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2"
Jul 08 19:32:38.847506 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.847484    2563 scope.go:117] "RemoveContainer" containerID="05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2"
Jul 08 19:32:38.847827 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:38.847805    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2\": container with ID starting with 05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2 not found: ID does not exist" containerID="05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2"
Jul 08 19:32:38.847868 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.847835    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2"} err="failed to get container status \"05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2\": rpc error: code = NotFound desc = could not find container \"05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2\": container with ID starting with 05d56f5526b8d5d3f78d7c16978fcbfc19d7a2b9142aa8096e98cf9d49a518b2 not found: ID does not exist"
Jul 08 19:32:38.859930 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.859890    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"]
Jul 08 19:32:38.863965 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.863935    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b30e0e2-5fc9d995d9-sz677"]
Jul 08 19:32:38.931060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:38.931022    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f995d345-1266-452d-968e-a6aa70ed215a" path="/var/lib/kubelet/pods/f995d345-1266-452d-968e-a6aa70ed215a/volumes"
Jul 08 19:32:43.837661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.837629    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:43.859375 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.859337    2563 generic.go:356] "Generic (PLEG): container finished" podID="3954dd64-692c-4498-bccb-378411c65fa8" containerID="acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a" exitCode=0
Jul 08 19:32:43.859575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.859408    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"
Jul 08 19:32:43.859575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.859427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk" event={"ID":"3954dd64-692c-4498-bccb-378411c65fa8","Type":"ContainerDied","Data":"acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a"}
Jul 08 19:32:43.859575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.859481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk" event={"ID":"3954dd64-692c-4498-bccb-378411c65fa8","Type":"ContainerDied","Data":"d289d4bfca306200cc94c266700867720c051767a65af40b6674c5912ca8d339"}
Jul 08 19:32:43.859575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.859518    2563 scope.go:117] "RemoveContainer" containerID="acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a"
Jul 08 19:32:43.862436 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.862412    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"]
Jul 08 19:32:43.862973 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.862921    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f995d345-1266-452d-968e-a6aa70ed215a" containerName="maas-api"
Jul 08 19:32:43.862973 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.862943    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f995d345-1266-452d-968e-a6aa70ed215a" containerName="maas-api"
Jul 08 19:32:43.862973 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.862971    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3954dd64-692c-4498-bccb-378411c65fa8" containerName="istio-proxy"
Jul 08 19:32:43.863194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.862981    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3954dd64-692c-4498-bccb-378411c65fa8" containerName="istio-proxy"
Jul 08 19:32:43.863194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.863163    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f995d345-1266-452d-968e-a6aa70ed215a" containerName="maas-api"
Jul 08 19:32:43.863194 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.863179    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3954dd64-692c-4498-bccb-378411c65fa8" containerName="istio-proxy"
Jul 08 19:32:43.865731 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.865709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.871049 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.871000    2563 scope.go:117] "RemoveContainer" containerID="acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a"
Jul 08 19:32:43.871448 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:43.871428    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a\": container with ID starting with acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a not found: ID does not exist" containerID="acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a"
Jul 08 19:32:43.871526 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.871459    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a"} err="failed to get container status \"acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a\": rpc error: code = NotFound desc = could not find container \"acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a\": container with ID starting with acfaa8329a603f380f8a6118cbb7a718796ed369de09835d02c30ec5decea10a not found: ID does not exist"
Jul 08 19:32:43.873653 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:43.873622    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-ait-53af5b6b-openshift-default-dockercfg-dt6df\" is forbidden: User \"system:node:ip-10-0-130-212.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-212.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-ait-53af5b6b-openshift-default-dockercfg-dt6df\"" type="*v1.Secret"
Jul 08 19:32:43.901331 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.901252    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"]
Jul 08 19:32:43.921163 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlqm4\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-kube-api-access-wlqm4\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921163 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-data\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921391 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921190    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3954dd64-692c-4498-bccb-378411c65fa8-istio-podinfo\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921391 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921230    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-envoy\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921391 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921268    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-socket\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921391 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921291    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-certs\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921391 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3954dd64-692c-4498-bccb-378411c65fa8-istiod-ca-cert\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-istio-token\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921451    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-credential-socket\") pod \"3954dd64-692c-4498-bccb-378411c65fa8\" (UID: \"3954dd64-692c-4498-bccb-378411c65fa8\") "
Jul 08 19:32:43.921631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istiod-ca-cert\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921784 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwvz9\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-kube-api-access-nwvz9\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921784 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-token\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921784 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921696    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:43.921784 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-data\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-podinfo\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921871    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3954dd64-692c-4498-bccb-378411c65fa8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:32:43.921987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-envoy\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-certs\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.921987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.921971    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-data" (OuterVolumeSpecName: "istio-data") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:43.922260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.922014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-socket\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.922260 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.922247    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:43.922539 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.922331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-credential-socket\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:43.922955 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.922913    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:43.923670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.923647    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3954dd64-692c-4498-bccb-378411c65fa8-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:43.923670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.923678    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:43.923670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.923692    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:43.923670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.923705    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:43.923670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.930873    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3954dd64-692c-4498-bccb-378411c65fa8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:32:43.923670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.931239    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-istio-token" (OuterVolumeSpecName: "istio-token") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:32:43.931894 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.931820    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:43.931894 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:43.931825    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-kube-api-access-wlqm4" (OuterVolumeSpecName: "kube-api-access-wlqm4") pod "3954dd64-692c-4498-bccb-378411c65fa8" (UID: "3954dd64-692c-4498-bccb-378411c65fa8"). InnerVolumeSpecName "kube-api-access-wlqm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:32:44.025010 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.024967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istiod-ca-cert\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025010 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwvz9\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-kube-api-access-nwvz9\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025290 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-token\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025290 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-data\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025290 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-podinfo\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-envoy\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-certs\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-socket\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-credential-socket\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025568    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:44.025613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025570    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-data\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025584    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:44.025613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025599    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlqm4\" (UniqueName: \"kubernetes.io/projected/3954dd64-692c-4498-bccb-378411c65fa8-kube-api-access-wlqm4\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:44.025613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025613    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3954dd64-692c-4498-bccb-378411c65fa8-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:44.025842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025627    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3954dd64-692c-4498-bccb-378411c65fa8-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:44.025842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025741    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-certs\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025842 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025794    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-socket\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025963 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025851    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istiod-ca-cert\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.025963 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.025869    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-credential-socket\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.027724 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.027694    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-envoy\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.027931 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.027915    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-podinfo\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.039633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.039604    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-token\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.048530 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.048504    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwvz9\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-kube-api-access-nwvz9\") pod \"e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") " pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:44.183133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.183092    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"]
Jul 08 19:32:44.190031 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.189992    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7b30e0e2-openshift-default-6fdfd94c-cc9vk"]
Jul 08 19:32:44.930680 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:44.930643    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3954dd64-692c-4498-bccb-378411c65fa8" path="/var/lib/kubelet/pods/3954dd64-692c-4498-bccb-378411c65fa8/volumes"
Jul 08 19:32:45.154915 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.154885    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"]
Jul 08 19:32:45.166210 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.166178    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.169514 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.169484    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:32:45.169677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.169586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"]
Jul 08 19:32:45.169923 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.169758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-53af5b6b-serving-cert\""
Jul 08 19:32:45.180980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.180910    2563 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 08 19:32:45.181154 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.180989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:45.236219 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.236184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.236392 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.236316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5bb971a-c29e-4783-aff3-72a7826e3e65-openshift-service-ca\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.236447 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.236395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9sf\" (UniqueName: \"kubernetes.io/projected/d5bb971a-c29e-4783-aff3-72a7826e3e65-kube-api-access-4t9sf\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.337397 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.337355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5bb971a-c29e-4783-aff3-72a7826e3e65-openshift-service-ca\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.337586 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.337439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9sf\" (UniqueName: \"kubernetes.io/projected/d5bb971a-c29e-4783-aff3-72a7826e3e65-kube-api-access-4t9sf\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.337586 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.337487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.337680 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:45.337637    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-serving-cert: secret "maas-api-e2e-ait-53af5b6b-serving-cert" not found
Jul 08 19:32:45.337721 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:45.337710    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls podName:d5bb971a-c29e-4783-aff3-72a7826e3e65 nodeName:}" failed. No retries permitted until 2026-07-08 19:32:45.837689186 +0000 UTC m=+1639.502251715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls") pod "maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" (UID: "d5bb971a-c29e-4783-aff3-72a7826e3e65") : secret "maas-api-e2e-ait-53af5b6b-serving-cert" not found
Jul 08 19:32:45.338158 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.338136    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5bb971a-c29e-4783-aff3-72a7826e3e65-openshift-service-ca\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.347176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.347152    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9sf\" (UniqueName: \"kubernetes.io/projected/d5bb971a-c29e-4783-aff3-72a7826e3e65-kube-api-access-4t9sf\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.352923 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.352899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"]
Jul 08 19:32:45.354747 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:32:45.354716    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3bfd5b_b154_422a_8cdf_abcd19afbb74.slice/crio-4f4cc817589e4e73b2c89217db0c71e5f38e17c879bfe56f312c000813067e69 WatchSource:0}: Error finding container 4f4cc817589e4e73b2c89217db0c71e5f38e17c879bfe56f312c000813067e69: Status 404 returned error can't find the container with id 4f4cc817589e4e73b2c89217db0c71e5f38e17c879bfe56f312c000813067e69
Jul 08 19:32:45.357478 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.357445    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:32:45.357551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.357515    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:32:45.357551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.357543    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:32:45.383744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.383716    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-53af5b6b-openshift-default-dockercfg-dt6df\""
Jul 08 19:32:45.842463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.842431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.845054 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.845030    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls\") pod \"maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:45.872632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.872593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" event={"ID":"9d3bfd5b-b154-422a-8cdf-abcd19afbb74","Type":"ContainerStarted","Data":"5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68"}
Jul 08 19:32:45.872803 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.872653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" event={"ID":"9d3bfd5b-b154-422a-8cdf-abcd19afbb74","Type":"ContainerStarted","Data":"4f4cc817589e4e73b2c89217db0c71e5f38e17c879bfe56f312c000813067e69"}
Jul 08 19:32:45.904740 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:45.904685    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" podStartSLOduration=2.904669077 podStartE2EDuration="2.904669077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:45.901681306 +0000 UTC m=+1639.566243858" watchObservedRunningTime="2026-07-08 19:32:45.904669077 +0000 UTC m=+1639.569231629"
Jul 08 19:32:46.080976 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.080943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:46.181476 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.181446    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:46.187985 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.187954    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:46.227101 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.227046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"]
Jul 08 19:32:46.228547 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:32:46.228522    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5bb971a_c29e_4783_aff3_72a7826e3e65.slice/crio-c9de91801b1f3a64b45dfdf2c30d0d5409c976cb9ee3c0e0516d9f36b2f20e58 WatchSource:0}: Error finding container c9de91801b1f3a64b45dfdf2c30d0d5409c976cb9ee3c0e0516d9f36b2f20e58: Status 404 returned error can't find the container with id c9de91801b1f3a64b45dfdf2c30d0d5409c976cb9ee3c0e0516d9f36b2f20e58
Jul 08 19:32:46.879016 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.878966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" event={"ID":"d5bb971a-c29e-4783-aff3-72a7826e3e65","Type":"ContainerStarted","Data":"ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621"}
Jul 08 19:32:46.879240 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.879042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" event={"ID":"d5bb971a-c29e-4783-aff3-72a7826e3e65","Type":"ContainerStarted","Data":"c9de91801b1f3a64b45dfdf2c30d0d5409c976cb9ee3c0e0516d9f36b2f20e58"}
Jul 08 19:32:46.879240 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.879209    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:46.879368 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.879355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:46.880345 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.880325    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:46.896431 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:46.896374    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" podStartSLOduration=1.562217741 podStartE2EDuration="1.896355411s" totalImagesPullingTime="334.13767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:46.896212633 +0000 UTC m=+1640.560775185" watchObservedRunningTime="2026-07-08 19:32:46.896355411 +0000 UTC m=+1640.560917961"
Jul 08 19:32:50.537917 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.537874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"]
Jul 08 19:32:50.538515 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.538182    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" podUID="d5bb971a-c29e-4783-aff3-72a7826e3e65" containerName="maas-api" containerID="cri-o://ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621" gracePeriod=30
Jul 08 19:32:50.804416 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.804388    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:50.897025 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.896991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls\") pod \"d5bb971a-c29e-4783-aff3-72a7826e3e65\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") "
Jul 08 19:32:50.897025 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.897032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t9sf\" (UniqueName: \"kubernetes.io/projected/d5bb971a-c29e-4783-aff3-72a7826e3e65-kube-api-access-4t9sf\") pod \"d5bb971a-c29e-4783-aff3-72a7826e3e65\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") "
Jul 08 19:32:50.897318 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.897160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5bb971a-c29e-4783-aff3-72a7826e3e65-openshift-service-ca\") pod \"d5bb971a-c29e-4783-aff3-72a7826e3e65\" (UID: \"d5bb971a-c29e-4783-aff3-72a7826e3e65\") "
Jul 08 19:32:50.897570 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.897538    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5bb971a-c29e-4783-aff3-72a7826e3e65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d5bb971a-c29e-4783-aff3-72a7826e3e65" (UID: "d5bb971a-c29e-4783-aff3-72a7826e3e65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:32:50.899457 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.899429    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d5bb971a-c29e-4783-aff3-72a7826e3e65" (UID: "d5bb971a-c29e-4783-aff3-72a7826e3e65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:32:50.899573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.899484    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5bb971a-c29e-4783-aff3-72a7826e3e65-kube-api-access-4t9sf" (OuterVolumeSpecName: "kube-api-access-4t9sf") pod "d5bb971a-c29e-4783-aff3-72a7826e3e65" (UID: "d5bb971a-c29e-4783-aff3-72a7826e3e65"). InnerVolumeSpecName "kube-api-access-4t9sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:32:50.903137 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.903110    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5bb971a-c29e-4783-aff3-72a7826e3e65" containerID="ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621" exitCode=0
Jul 08 19:32:50.903261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.903152    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"
Jul 08 19:32:50.903261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.903191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" event={"ID":"d5bb971a-c29e-4783-aff3-72a7826e3e65","Type":"ContainerDied","Data":"ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621"}
Jul 08 19:32:50.903261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.903248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7" event={"ID":"d5bb971a-c29e-4783-aff3-72a7826e3e65","Type":"ContainerDied","Data":"c9de91801b1f3a64b45dfdf2c30d0d5409c976cb9ee3c0e0516d9f36b2f20e58"}
Jul 08 19:32:50.903394 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.903274    2563 scope.go:117] "RemoveContainer" containerID="ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621"
Jul 08 19:32:50.922644 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.922624    2563 scope.go:117] "RemoveContainer" containerID="ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621"
Jul 08 19:32:50.922983 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:50.922955    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621\": container with ID starting with ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621 not found: ID does not exist" containerID="ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621"
Jul 08 19:32:50.923022 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.922996    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621"} err="failed to get container status \"ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621\": rpc error: code = NotFound desc = could not find container \"ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621\": container with ID starting with ab878a181aa863408429d33a38b891e5eba0be377c21554aaf2568ace1b9c621 not found: ID does not exist"
Jul 08 19:32:50.932235 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.932202    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"]
Jul 08 19:32:50.936748 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.936715    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-53af5b6b-84f64bb557-fvgx7"]
Jul 08 19:32:50.998437 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.998399    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5bb971a-c29e-4783-aff3-72a7826e3e65-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:50.998437 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.998433    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5bb971a-c29e-4783-aff3-72a7826e3e65-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:50.998437 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:50.998445    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t9sf\" (UniqueName: \"kubernetes.io/projected/d5bb971a-c29e-4783-aff3-72a7826e3e65-kube-api-access-4t9sf\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:52.930766 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:52.930735    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5bb971a-c29e-4783-aff3-72a7826e3e65" path="/var/lib/kubelet/pods/d5bb971a-c29e-4783-aff3-72a7826e3e65/volumes"
Jul 08 19:32:56.345674 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:56.345577    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"]
Jul 08 19:32:56.346122 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:56.345819    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" podUID="9d3bfd5b-b154-422a-8cdf-abcd19afbb74" containerName="istio-proxy" containerID="cri-o://5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68" gracePeriod=30
Jul 08 19:32:58.742590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.742560    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:58.883987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.883949    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istiod-ca-cert\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.883987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.883997    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-data\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884026    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-token\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884047    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-socket\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884238    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-credential-socket\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884439 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884322    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:58.884439 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884364    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-certs\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884453    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:32:58.884551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwvz9\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-kube-api-access-nwvz9\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884502    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-podinfo\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884540    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-envoy\") pod \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\" (UID: \"9d3bfd5b-b154-422a-8cdf-abcd19afbb74\") "
Jul 08 19:32:58.884742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884586    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:58.884810 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884774    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:58.885006 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.884985    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.885101 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.885011    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.885101 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.885025    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.885101 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.885042    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.885428 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.885182    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:58.887947 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.887903    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:32:58.887947 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.887918    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:32:58.888135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.887989    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-kube-api-access-nwvz9" (OuterVolumeSpecName: "kube-api-access-nwvz9") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "kube-api-access-nwvz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:32:58.888135 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.887993    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d3bfd5b-b154-422a-8cdf-abcd19afbb74" (UID: "9d3bfd5b-b154-422a-8cdf-abcd19afbb74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:32:58.948163 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.948118    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d3bfd5b-b154-422a-8cdf-abcd19afbb74" containerID="5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68" exitCode=0
Jul 08 19:32:58.948323 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.948170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" event={"ID":"9d3bfd5b-b154-422a-8cdf-abcd19afbb74","Type":"ContainerDied","Data":"5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68"}
Jul 08 19:32:58.948323 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.948214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn" event={"ID":"9d3bfd5b-b154-422a-8cdf-abcd19afbb74","Type":"ContainerDied","Data":"4f4cc817589e4e73b2c89217db0c71e5f38e17c879bfe56f312c000813067e69"}
Jul 08 19:32:58.948323 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.948237    2563 scope.go:117] "RemoveContainer" containerID="5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68"
Jul 08 19:32:58.948323 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.948246    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"
Jul 08 19:32:58.957978 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.957957    2563 scope.go:117] "RemoveContainer" containerID="5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68"
Jul 08 19:32:58.958338 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:32:58.958317    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68\": container with ID starting with 5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68 not found: ID does not exist" containerID="5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68"
Jul 08 19:32:58.958405 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.958348    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68"} err="failed to get container status \"5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68\": rpc error: code = NotFound desc = could not find container \"5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68\": container with ID starting with 5f0b5e8f4b8ef6fab4365a4a4b84f9404fa5bb164ca59edb6390a056d312ab68 not found: ID does not exist"
Jul 08 19:32:58.967021 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.966991    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"]
Jul 08 19:32:58.971037 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.971009    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-53af5b6b-openshift-default-596dc87675-qthcn"]
Jul 08 19:32:58.986509 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.986470    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.986509 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.986503    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.986707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.986518    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwvz9\" (UniqueName: \"kubernetes.io/projected/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-kube-api-access-nwvz9\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.986707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.986531    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:32:58.986707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:32:58.986545    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d3bfd5b-b154-422a-8cdf-abcd19afbb74-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:00.931303 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:00.931273    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d3bfd5b-b154-422a-8cdf-abcd19afbb74" path="/var/lib/kubelet/pods/9d3bfd5b-b154-422a-8cdf-abcd19afbb74/volumes"
Jul 08 19:33:02.895427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.895379    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"]
Jul 08 19:33:02.895986 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.895967    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3bfd5b-b154-422a-8cdf-abcd19afbb74" containerName="istio-proxy"
Jul 08 19:33:02.896044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.895991    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3bfd5b-b154-422a-8cdf-abcd19afbb74" containerName="istio-proxy"
Jul 08 19:33:02.896044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.896013    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5bb971a-c29e-4783-aff3-72a7826e3e65" containerName="maas-api"
Jul 08 19:33:02.896044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.896022    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5bb971a-c29e-4783-aff3-72a7826e3e65" containerName="maas-api"
Jul 08 19:33:02.896211 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.896141    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5bb971a-c29e-4783-aff3-72a7826e3e65" containerName="maas-api"
Jul 08 19:33:02.896211 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.896154    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d3bfd5b-b154-422a-8cdf-abcd19afbb74" containerName="istio-proxy"
Jul 08 19:33:02.899717 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.899695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:02.903454 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.903411    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-65eb7314-openshift-default-dockercfg-948nz\""
Jul 08 19:33:02.920156 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:02.920120    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"]
Jul 08 19:33:03.024584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-envoy\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-credential-socket\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024795 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df238a17-f650-4320-b2a1-d6fe3e4d9120-istiod-ca-cert\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024795 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-podinfo\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024795 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h99tv\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-kube-api-access-h99tv\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024795 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-certs\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024795 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-socket\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024962 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-data\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.024962 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.024907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-token\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126475 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-envoy\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126475 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-credential-socket\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126745 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df238a17-f650-4320-b2a1-d6fe3e4d9120-istiod-ca-cert\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126745 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-podinfo\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126745 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h99tv\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-kube-api-access-h99tv\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126745 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-certs\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-socket\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-data\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.126949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.126908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-token\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.127140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.127036    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-credential-socket\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.127208 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.127132    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-socket\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.127208 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.127183    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-certs\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.127556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.127536    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-data\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.127613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.127598    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df238a17-f650-4320-b2a1-d6fe3e4d9120-istiod-ca-cert\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.129129 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.129077    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-envoy\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.129366 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.129344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-podinfo\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.135379 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.135350    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-token\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.135496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.135417    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h99tv\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-kube-api-access-h99tv\") pod \"e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") " pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.213056 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.212954    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:03.368355 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.368317    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"]
Jul 08 19:33:03.371972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.371937    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:33:03.372142 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.372008    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:33:03.372142 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.372034    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:33:03.976305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.976258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk" event={"ID":"df238a17-f650-4320-b2a1-d6fe3e4d9120","Type":"ContainerStarted","Data":"792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7"}
Jul 08 19:33:03.976715 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:03.976330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk" event={"ID":"df238a17-f650-4320-b2a1-d6fe3e4d9120","Type":"ContainerStarted","Data":"eeccb2d7c82de1373b8e7de42cb49e1590022bc56f67f95e4c5c65ebb806495e"}
Jul 08 19:33:04.002050 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.001989    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk" podStartSLOduration=2.001968484 podStartE2EDuration="2.001968484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:33:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:03.999618277 +0000 UTC m=+1657.664180830" watchObservedRunningTime="2026-07-08 19:33:04.001968484 +0000 UTC m=+1657.666531034"
Jul 08 19:33:04.147352 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.147315    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"]
Jul 08 19:33:04.150719 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.150679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.153267 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.153236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-65eb7314-serving-cert\""
Jul 08 19:33:04.153404 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.153269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:33:04.162392 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.162363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"]
Jul 08 19:33:04.213439 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.213402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:04.220183 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.220143    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:04.340855 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.340759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c0a2bd-923a-478e-93a5-525ad5314b07-maas-api-tls\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.340855 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.340840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prcjq\" (UniqueName: \"kubernetes.io/projected/d6c0a2bd-923a-478e-93a5-525ad5314b07-kube-api-access-prcjq\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.341050 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.340869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c0a2bd-923a-478e-93a5-525ad5314b07-openshift-service-ca\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.442430 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.442386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c0a2bd-923a-478e-93a5-525ad5314b07-maas-api-tls\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.442623 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.442463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prcjq\" (UniqueName: \"kubernetes.io/projected/d6c0a2bd-923a-478e-93a5-525ad5314b07-kube-api-access-prcjq\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.442623 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.442483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c0a2bd-923a-478e-93a5-525ad5314b07-openshift-service-ca\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.443188 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.443170    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c0a2bd-923a-478e-93a5-525ad5314b07-openshift-service-ca\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.445132 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.445070    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c0a2bd-923a-478e-93a5-525ad5314b07-maas-api-tls\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.451209 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.451183    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prcjq\" (UniqueName: \"kubernetes.io/projected/d6c0a2bd-923a-478e-93a5-525ad5314b07-kube-api-access-prcjq\") pod \"maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.464653 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.464622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:04.615140 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.615109    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"]
Jul 08 19:33:04.616870 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:33:04.616839    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c0a2bd_923a_478e_93a5_525ad5314b07.slice/crio-5bd81c4fa973d331125a3fe6cd40924471e3d80a7ef798baf1dd8857af101574 WatchSource:0}: Error finding container 5bd81c4fa973d331125a3fe6cd40924471e3d80a7ef798baf1dd8857af101574: Status 404 returned error can't find the container with id 5bd81c4fa973d331125a3fe6cd40924471e3d80a7ef798baf1dd8857af101574
Jul 08 19:33:04.983097 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.983047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s" event={"ID":"d6c0a2bd-923a-478e-93a5-525ad5314b07","Type":"ContainerStarted","Data":"5bd81c4fa973d331125a3fe6cd40924471e3d80a7ef798baf1dd8857af101574"}
Jul 08 19:33:04.983463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.983321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:04.984580 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:04.984562    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:05.990041 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:05.990001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s" event={"ID":"d6c0a2bd-923a-478e-93a5-525ad5314b07","Type":"ContainerStarted","Data":"5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914"}
Jul 08 19:33:05.990484 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:05.990099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:06.009421 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:06.009365    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s" podStartSLOduration=1.652303557 podStartE2EDuration="2.009346228s" totalImagesPullingTime="357.042671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:33:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:06.007310231 +0000 UTC m=+1659.671872784" watchObservedRunningTime="2026-07-08 19:33:06.009346228 +0000 UTC m=+1659.673908779"
Jul 08 19:33:08.726269 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:08.726222    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"]
Jul 08 19:33:08.726765 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:08.726485    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk" podUID="df238a17-f650-4320-b2a1-d6fe3e4d9120" containerName="istio-proxy" containerID="cri-o://792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7" gracePeriod=30
Jul 08 19:33:08.943104 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:08.943049    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"]
Jul 08 19:33:08.943555 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:08.943501    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s" podUID="d6c0a2bd-923a-478e-93a5-525ad5314b07" containerName="maas-api" containerID="cri-o://5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914" gracePeriod=30
Jul 08 19:33:09.255586 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.255559    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:09.296959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.295474    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c0a2bd-923a-478e-93a5-525ad5314b07-maas-api-tls\") pod \"d6c0a2bd-923a-478e-93a5-525ad5314b07\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") "
Jul 08 19:33:09.296959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.295586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prcjq\" (UniqueName: \"kubernetes.io/projected/d6c0a2bd-923a-478e-93a5-525ad5314b07-kube-api-access-prcjq\") pod \"d6c0a2bd-923a-478e-93a5-525ad5314b07\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") "
Jul 08 19:33:09.296959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.295630    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c0a2bd-923a-478e-93a5-525ad5314b07-openshift-service-ca\") pod \"d6c0a2bd-923a-478e-93a5-525ad5314b07\" (UID: \"d6c0a2bd-923a-478e-93a5-525ad5314b07\") "
Jul 08 19:33:09.296959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.296678    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6c0a2bd-923a-478e-93a5-525ad5314b07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6c0a2bd-923a-478e-93a5-525ad5314b07" (UID: "d6c0a2bd-923a-478e-93a5-525ad5314b07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:33:09.300429 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.300370    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6c0a2bd-923a-478e-93a5-525ad5314b07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6c0a2bd-923a-478e-93a5-525ad5314b07" (UID: "d6c0a2bd-923a-478e-93a5-525ad5314b07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:33:09.307351 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.307257    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c0a2bd-923a-478e-93a5-525ad5314b07-kube-api-access-prcjq" (OuterVolumeSpecName: "kube-api-access-prcjq") pod "d6c0a2bd-923a-478e-93a5-525ad5314b07" (UID: "d6c0a2bd-923a-478e-93a5-525ad5314b07"). InnerVolumeSpecName "kube-api-access-prcjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:33:09.403742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.403687    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c0a2bd-923a-478e-93a5-525ad5314b07-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:09.403742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.403727    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prcjq\" (UniqueName: \"kubernetes.io/projected/d6c0a2bd-923a-478e-93a5-525ad5314b07-kube-api-access-prcjq\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:09.403742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:09.403744    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c0a2bd-923a-478e-93a5-525ad5314b07-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:10.017115 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.017064    2563 generic.go:356] "Generic (PLEG): container finished" podID="d6c0a2bd-923a-478e-93a5-525ad5314b07" containerID="5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914" exitCode=0
Jul 08 19:33:10.017603 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.017115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s" event={"ID":"d6c0a2bd-923a-478e-93a5-525ad5314b07","Type":"ContainerDied","Data":"5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914"}
Jul 08 19:33:10.017603 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.017136    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"
Jul 08 19:33:10.017603 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.017175    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s" event={"ID":"d6c0a2bd-923a-478e-93a5-525ad5314b07","Type":"ContainerDied","Data":"5bd81c4fa973d331125a3fe6cd40924471e3d80a7ef798baf1dd8857af101574"}
Jul 08 19:33:10.017603 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.017198    2563 scope.go:117] "RemoveContainer" containerID="5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914"
Jul 08 19:33:10.026989 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.026967    2563 scope.go:117] "RemoveContainer" containerID="5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914"
Jul 08 19:33:10.027325 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:33:10.027302    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914\": container with ID starting with 5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914 not found: ID does not exist" containerID="5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914"
Jul 08 19:33:10.027394 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.027337    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914"} err="failed to get container status \"5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914\": rpc error: code = NotFound desc = could not find container \"5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914\": container with ID starting with 5a0af3abe6a74906c8784eab6dc41cd560e89586163c453d7f21710aa45b6914 not found: ID does not exist"
Jul 08 19:33:10.039832 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.039795    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"]
Jul 08 19:33:10.043308 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.043281    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65eb7314-6c8fb4559f-z5p9s"]
Jul 08 19:33:10.931536 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:10.931496    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6c0a2bd-923a-478e-93a5-525ad5314b07" path="/var/lib/kubelet/pods/d6c0a2bd-923a-478e-93a5-525ad5314b07/volumes"
Jul 08 19:33:13.988954 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:13.988929    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:14.043876 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.043840    2563 generic.go:356] "Generic (PLEG): container finished" podID="df238a17-f650-4320-b2a1-d6fe3e4d9120" containerID="792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7" exitCode=0
Jul 08 19:33:14.044053 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.043887    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"
Jul 08 19:33:14.044053 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.043922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk" event={"ID":"df238a17-f650-4320-b2a1-d6fe3e4d9120","Type":"ContainerDied","Data":"792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7"}
Jul 08 19:33:14.044053 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.043973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk" event={"ID":"df238a17-f650-4320-b2a1-d6fe3e4d9120","Type":"ContainerDied","Data":"eeccb2d7c82de1373b8e7de42cb49e1590022bc56f67f95e4c5c65ebb806495e"}
Jul 08 19:33:14.044053 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.043995    2563 scope.go:117] "RemoveContainer" containerID="792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7"
Jul 08 19:33:14.053005 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.052985    2563 scope.go:117] "RemoveContainer" containerID="792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7"
Jul 08 19:33:14.053359 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:33:14.053338    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7\": container with ID starting with 792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7 not found: ID does not exist" containerID="792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7"
Jul 08 19:33:14.053416 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053369    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7"} err="failed to get container status \"792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7\": rpc error: code = NotFound desc = could not find container \"792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7\": container with ID starting with 792b6010aadcdd30b9608fbf7f23bd0d664ae4d09e3eb228a312e38488879ef7 not found: ID does not exist"
Jul 08 19:33:14.053416 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-token\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053494 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053439    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-credential-socket\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053494 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053475    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-envoy\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053494 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053491    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df238a17-f650-4320-b2a1-d6fe3e4d9120-istiod-ca-cert\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053630 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-socket\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053686 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053670    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-certs\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053743 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h99tv\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-kube-api-access-h99tv\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.053844 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053824    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:33:14.053911 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053821    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:33:14.053993 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.053965    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df238a17-f650-4320-b2a1-d6fe3e4d9120-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:33:14.054320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054002    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:33:14.054320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-podinfo\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.054320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-data\") pod \"df238a17-f650-4320-b2a1-d6fe3e4d9120\" (UID: \"df238a17-f650-4320-b2a1-d6fe3e4d9120\") "
Jul 08 19:33:14.054516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054401    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df238a17-f650-4320-b2a1-d6fe3e4d9120-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.054516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054496    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.054516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054511    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.054677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.054525    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.055236 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.055199    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-data" (OuterVolumeSpecName: "istio-data") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:33:14.055924 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.055901    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-token" (OuterVolumeSpecName: "istio-token") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:33:14.056181 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.056063    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:33:14.056181 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.056131    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:33:14.056285 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.056189    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-kube-api-access-h99tv" (OuterVolumeSpecName: "kube-api-access-h99tv") pod "df238a17-f650-4320-b2a1-d6fe3e4d9120" (UID: "df238a17-f650-4320-b2a1-d6fe3e4d9120"). InnerVolumeSpecName "kube-api-access-h99tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:33:14.155335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.155300    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h99tv\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-kube-api-access-h99tv\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.155335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.155332    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.155335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.155344    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.155588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.155353    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.155588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.155361    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df238a17-f650-4320-b2a1-d6fe3e4d9120-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:33:14.367874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.367844    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"]
Jul 08 19:33:14.372372 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.372343    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-65eb7314-openshift-default-bc749cb95-t6lkk"]
Jul 08 19:33:14.931969 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:33:14.931922    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df238a17-f650-4320-b2a1-d6fe3e4d9120" path="/var/lib/kubelet/pods/df238a17-f650-4320-b2a1-d6fe3e4d9120/volumes"
Jul 08 19:34:51.952631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.952599    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"]
Jul 08 19:34:51.953044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.953022    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df238a17-f650-4320-b2a1-d6fe3e4d9120" containerName="istio-proxy"
Jul 08 19:34:51.953044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.953037    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df238a17-f650-4320-b2a1-d6fe3e4d9120" containerName="istio-proxy"
Jul 08 19:34:51.953145 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.953050    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c0a2bd-923a-478e-93a5-525ad5314b07" containerName="maas-api"
Jul 08 19:34:51.953145 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.953055    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c0a2bd-923a-478e-93a5-525ad5314b07" containerName="maas-api"
Jul 08 19:34:51.953145 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.953137    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df238a17-f650-4320-b2a1-d6fe3e4d9120" containerName="istio-proxy"
Jul 08 19:34:51.953145 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.953146    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6c0a2bd-923a-478e-93a5-525ad5314b07" containerName="maas-api"
Jul 08 19:34:51.955498 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.955479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:51.958336 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.958312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a559bc20-openshift-default-dockercfg-nzzgq\""
Jul 08 19:34:51.982035 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:51.981985    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"]
Jul 08 19:34:52.099506 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.099464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istiod-ca-cert\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.099783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.099762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-certs\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.099903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.099888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-credential-socket\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.100032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.100019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-token\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.100209 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.100191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-envoy\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.100358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.100344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-socket\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.100463 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.100449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tjtr\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-kube-api-access-2tjtr\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.100576 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.100563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-data\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.100718 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.100704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-podinfo\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istiod-ca-cert\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-certs\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-credential-socket\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-token\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-envoy\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-socket\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tjtr\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-kube-api-access-2tjtr\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-data\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.202613 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-podinfo\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.203028 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202759    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-certs\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.203028 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.202823    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-credential-socket\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.203180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.203108    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-socket\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.203241 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.203180    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-data\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.203241 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.203183    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istiod-ca-cert\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.205185 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.205166    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-envoy\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.205251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.205226    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-podinfo\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.211165 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.211138    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-token\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.211165 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.211152    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tjtr\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-kube-api-access-2tjtr\") pod \"e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") " pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.266183 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.266145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:52.415978 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.415940    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"]
Jul 08 19:34:52.419344 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:34:52.419306    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2873d2d1_6beb_49ad_a7b1_eb9c34ab27ca.slice/crio-b68e65ea34f4a1aa075ba6edd34b9eecb1625d0f08dfc2da7f74c52881957303 WatchSource:0}: Error finding container b68e65ea34f4a1aa075ba6edd34b9eecb1625d0f08dfc2da7f74c52881957303: Status 404 returned error can't find the container with id b68e65ea34f4a1aa075ba6edd34b9eecb1625d0f08dfc2da7f74c52881957303
Jul 08 19:34:52.421672 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.421637    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:34:52.421780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.421711    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:34:52.421780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.421740    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:34:52.556337 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.556299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" event={"ID":"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca","Type":"ContainerStarted","Data":"584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"}
Jul 08 19:34:52.556548 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.556357    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" event={"ID":"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca","Type":"ContainerStarted","Data":"b68e65ea34f4a1aa075ba6edd34b9eecb1625d0f08dfc2da7f74c52881957303"}
Jul 08 19:34:52.583202 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:52.583141    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" podStartSLOduration=1.5831237219999998 podStartE2EDuration="1.583123722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:52.58077784 +0000 UTC m=+1766.245340392" watchObservedRunningTime="2026-07-08 19:34:52.583123722 +0000 UTC m=+1766.247686273"
Jul 08 19:34:53.267233 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:53.267202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:53.272025 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:53.272001    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:53.562195 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:53.562117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:53.563148 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:53.563130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:34:54.735699 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.735654    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"]
Jul 08 19:34:54.738947 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.738917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.742666 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.742530    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:34:54.742666 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.742594    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a559bc20-serving-cert\""
Jul 08 19:34:54.748703 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.748679    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"]
Jul 08 19:34:54.831690 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.831655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.831879 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.831729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f78b021-5929-454b-b76e-12e4fbca668c-openshift-service-ca\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.831879 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.831763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2vc8\" (UniqueName: \"kubernetes.io/projected/2f78b021-5929-454b-b76e-12e4fbca668c-kube-api-access-h2vc8\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.933167 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.933129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.933343 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.933209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f78b021-5929-454b-b76e-12e4fbca668c-openshift-service-ca\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.933343 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.933242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vc8\" (UniqueName: \"kubernetes.io/projected/2f78b021-5929-454b-b76e-12e4fbca668c-kube-api-access-h2vc8\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.933343 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:34:54.933291    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-serving-cert: secret "maas-api-e2e-mt-a559bc20-serving-cert" not found
Jul 08 19:34:54.933508 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:34:54.933373    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls podName:2f78b021-5929-454b-b76e-12e4fbca668c nodeName:}" failed. No retries permitted until 2026-07-08 19:34:55.433349403 +0000 UTC m=+1769.097911934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls") pod "maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" (UID: "2f78b021-5929-454b-b76e-12e4fbca668c") : secret "maas-api-e2e-mt-a559bc20-serving-cert" not found
Jul 08 19:34:54.934070 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.934039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f78b021-5929-454b-b76e-12e4fbca668c-openshift-service-ca\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:54.942903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:54.942872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vc8\" (UniqueName: \"kubernetes.io/projected/2f78b021-5929-454b-b76e-12e4fbca668c-kube-api-access-h2vc8\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:55.439311 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:55.439278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:55.442180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:55.442153    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls\") pod \"maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:55.654308 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:55.654267    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:55.800173 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:55.800147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"]
Jul 08 19:34:55.802360 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:34:55.802328    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f78b021_5929_454b_b76e_12e4fbca668c.slice/crio-3cc475beb48956ee8fa32a5f3b1e837b6b64baf87dc37abe4207049611c4308f WatchSource:0}: Error finding container 3cc475beb48956ee8fa32a5f3b1e837b6b64baf87dc37abe4207049611c4308f: Status 404 returned error can't find the container with id 3cc475beb48956ee8fa32a5f3b1e837b6b64baf87dc37abe4207049611c4308f
Jul 08 19:34:56.578036 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:56.577941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" event={"ID":"2f78b021-5929-454b-b76e-12e4fbca668c","Type":"ContainerStarted","Data":"8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4"}
Jul 08 19:34:56.578036 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:56.577998    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" event={"ID":"2f78b021-5929-454b-b76e-12e4fbca668c","Type":"ContainerStarted","Data":"3cc475beb48956ee8fa32a5f3b1e837b6b64baf87dc37abe4207049611c4308f"}
Jul 08 19:34:56.578036 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:56.578021    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:34:56.596972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:34:56.596913    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" podStartSLOduration=2.124353923 podStartE2EDuration="2.596896772s" totalImagesPullingTime="472.542849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:56.595697652 +0000 UTC m=+1770.260260201" watchObservedRunningTime="2026-07-08 19:34:56.596896772 +0000 UTC m=+1770.261459331"
Jul 08 19:35:02.595781 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:02.594993    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:35:06.636286 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:06.636252    2563 generic.go:356] "Generic (PLEG): container finished" podID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerID="584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03" exitCode=0
Jul 08 19:35:06.636722 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:06.636326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" event={"ID":"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca","Type":"ContainerDied","Data":"584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"}
Jul 08 19:35:06.636780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:06.636739    2563 scope.go:117] "RemoveContainer" containerID="584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"
Jul 08 19:35:06.637779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:06.637748    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:06.637839 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:06.637818    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:06.637878 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:06.637853    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:07.249798 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.249764    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"]
Jul 08 19:35:07.250032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.250003    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" podUID="2f78b021-5929-454b-b76e-12e4fbca668c" containerName="maas-api" containerID="cri-o://8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4" gracePeriod=30
Jul 08 19:35:07.527419 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.527391    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:35:07.587109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.584407    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls\") pod \"2f78b021-5929-454b-b76e-12e4fbca668c\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") "
Jul 08 19:35:07.587109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.584541    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f78b021-5929-454b-b76e-12e4fbca668c-openshift-service-ca\") pod \"2f78b021-5929-454b-b76e-12e4fbca668c\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") "
Jul 08 19:35:07.587109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.584630    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2vc8\" (UniqueName: \"kubernetes.io/projected/2f78b021-5929-454b-b76e-12e4fbca668c-kube-api-access-h2vc8\") pod \"2f78b021-5929-454b-b76e-12e4fbca668c\" (UID: \"2f78b021-5929-454b-b76e-12e4fbca668c\") "
Jul 08 19:35:07.591108 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.588519    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f78b021-5929-454b-b76e-12e4fbca668c" (UID: "2f78b021-5929-454b-b76e-12e4fbca668c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:35:07.595108 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.594294    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f78b021-5929-454b-b76e-12e4fbca668c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f78b021-5929-454b-b76e-12e4fbca668c" (UID: "2f78b021-5929-454b-b76e-12e4fbca668c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:35:07.605302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.605252    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f78b021-5929-454b-b76e-12e4fbca668c-kube-api-access-h2vc8" (OuterVolumeSpecName: "kube-api-access-h2vc8") pod "2f78b021-5929-454b-b76e-12e4fbca668c" (UID: "2f78b021-5929-454b-b76e-12e4fbca668c"). InnerVolumeSpecName "kube-api-access-h2vc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:35:07.643359 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.643321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" event={"ID":"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca","Type":"ContainerStarted","Data":"96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e"}
Jul 08 19:35:07.644580 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.644555    2563 generic.go:356] "Generic (PLEG): container finished" podID="2f78b021-5929-454b-b76e-12e4fbca668c" containerID="8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4" exitCode=0
Jul 08 19:35:07.644736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.644588    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"
Jul 08 19:35:07.644736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.644625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" event={"ID":"2f78b021-5929-454b-b76e-12e4fbca668c","Type":"ContainerDied","Data":"8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4"}
Jul 08 19:35:07.644736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.644679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4" event={"ID":"2f78b021-5929-454b-b76e-12e4fbca668c","Type":"ContainerDied","Data":"3cc475beb48956ee8fa32a5f3b1e837b6b64baf87dc37abe4207049611c4308f"}
Jul 08 19:35:07.644736 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.644709    2563 scope.go:117] "RemoveContainer" containerID="8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4"
Jul 08 19:35:07.653964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.653943    2563 scope.go:117] "RemoveContainer" containerID="8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4"
Jul 08 19:35:07.654275 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:35:07.654257    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4\": container with ID starting with 8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4 not found: ID does not exist" containerID="8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4"
Jul 08 19:35:07.654342 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.654284    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4"} err="failed to get container status \"8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4\": rpc error: code = NotFound desc = could not find container \"8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4\": container with ID starting with 8aa2bce9ca4918f9b347ebea3c7289463bfe5f8679c6b75843689ff3a539f9e4 not found: ID does not exist"
Jul 08 19:35:07.694109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.686442    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2vc8\" (UniqueName: \"kubernetes.io/projected/2f78b021-5929-454b-b76e-12e4fbca668c-kube-api-access-h2vc8\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:07.694109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.686475    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f78b021-5929-454b-b76e-12e4fbca668c-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:07.694109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.686492    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f78b021-5929-454b-b76e-12e4fbca668c-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:07.694109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.691158    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"]
Jul 08 19:35:07.694109 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:07.693725    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a559bc20-ccc5cdbd5-tvnl4"]
Jul 08 19:35:08.266566 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:08.266526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:35:08.267800 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:08.267765    2563 patch_prober.go:28] interesting pod/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 08 19:35:08.267923 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:08.267829    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 08 19:35:08.562490 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:08.562390    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:35:08.930000 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:08.929965    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f78b021-5929-454b-b76e-12e4fbca668c" path="/var/lib/kubelet/pods/2f78b021-5929-454b-b76e-12e4fbca668c/volumes"
Jul 08 19:35:09.267611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:09.267513    2563 patch_prober.go:28] interesting pod/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 08 19:35:09.267611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:09.267589    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 08 19:35:10.267433 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:10.267396    2563 patch_prober.go:28] interesting pod/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 08 19:35:10.267834 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:10.267457    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 08 19:35:11.266636 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:11.266593    2563 patch_prober.go:28] interesting pod/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 08 19:35:11.266838 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:11.266671    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 08 19:35:12.270623 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:12.270594    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:35:12.271632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:12.271610    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:35:13.737050 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:13.737009    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"]
Jul 08 19:35:13.737472 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:13.737324    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy" containerID="cri-o://96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e" gracePeriod=30
Jul 08 19:35:19.003105 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.003056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:35:19.112846 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.112813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-certs\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.112882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-data\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.112917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istiod-ca-cert\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.112937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-socket\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.112963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-envoy\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-podinfo\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113063    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tjtr\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-kube-api-access-2tjtr\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-credential-socket\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113162    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-token\") pod \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\" (UID: \"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca\") "
Jul 08 19:35:19.113400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113179    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:35:19.113400 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113376    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:35:19.113709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113441    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:35:19.113709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113597    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:35:19.113709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113662    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.113709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113685    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.113709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.113700    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.114045 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.114020    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:35:19.115485 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.115466    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:35:19.115669 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.115646    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:35:19.116005 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.115979    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:35:19.116122 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.115997    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-kube-api-access-2tjtr" (OuterVolumeSpecName: "kube-api-access-2tjtr") pod "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" (UID: "2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca"). InnerVolumeSpecName "kube-api-access-2tjtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:35:19.214631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.214582    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.214631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.214621    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tjtr\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-kube-api-access-2tjtr\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.214631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.214632    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.214631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.214640    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.214915 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.214648    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.214915 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.214657    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:35:19.708238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.708195    2563 generic.go:356] "Generic (PLEG): container finished" podID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerID="96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e" exitCode=0
Jul 08 19:35:19.708414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.708247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"
Jul 08 19:35:19.708414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.708280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" event={"ID":"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca","Type":"ContainerDied","Data":"96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e"}
Jul 08 19:35:19.708414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.708332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7" event={"ID":"2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca","Type":"ContainerDied","Data":"b68e65ea34f4a1aa075ba6edd34b9eecb1625d0f08dfc2da7f74c52881957303"}
Jul 08 19:35:19.708414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.708354    2563 scope.go:117] "RemoveContainer" containerID="96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e"
Jul 08 19:35:19.718352 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.718329    2563 scope.go:117] "RemoveContainer" containerID="584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"
Jul 08 19:35:19.726958 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.726936    2563 scope.go:117] "RemoveContainer" containerID="96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e"
Jul 08 19:35:19.727376 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:35:19.727271    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e\": container with ID starting with 96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e not found: ID does not exist" containerID="96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e"
Jul 08 19:35:19.727376 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.727313    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e"} err="failed to get container status \"96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e\": rpc error: code = NotFound desc = could not find container \"96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e\": container with ID starting with 96e6b1fbab9707b77873a81664a3c1b68ac99584505671666a55169f1e70b81e not found: ID does not exist"
Jul 08 19:35:19.727376 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.727340    2563 scope.go:117] "RemoveContainer" containerID="584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"
Jul 08 19:35:19.727953 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:35:19.727907    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03\": container with ID starting with 584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03 not found: ID does not exist" containerID="584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"
Jul 08 19:35:19.728044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.727946    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03"} err="failed to get container status \"584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03\": rpc error: code = NotFound desc = could not find container \"584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03\": container with ID starting with 584596821e5a77b69f9e6eb65f47eda57825d1043541e9584df99c92d0694d03 not found: ID does not exist"
Jul 08 19:35:19.730374 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.730353    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"]
Jul 08 19:35:19.733851 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:19.733827    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a559bc20-openshift-default-6cd5bbcfcf-zbqb7"]
Jul 08 19:35:20.931130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:20.931073    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" path="/var/lib/kubelet/pods/2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca/volumes"
Jul 08 19:35:34.048869 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.048818    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"]
Jul 08 19:35:34.049451 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049435    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f78b021-5929-454b-b76e-12e4fbca668c" containerName="maas-api"
Jul 08 19:35:34.049497 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049456    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f78b021-5929-454b-b76e-12e4fbca668c" containerName="maas-api"
Jul 08 19:35:34.049497 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049482    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy"
Jul 08 19:35:34.049497 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049491    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy"
Jul 08 19:35:34.049595 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049502    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy"
Jul 08 19:35:34.049595 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049511    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy"
Jul 08 19:35:34.049662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049633    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f78b021-5929-454b-b76e-12e4fbca668c" containerName="maas-api"
Jul 08 19:35:34.049662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.049649    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy"
Jul 08 19:35:34.052561 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.052542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.055890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.055866    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3f1b3726-openshift-default-dockercfg-wgll6\""
Jul 08 19:35:34.056377 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056356    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-credential-socket\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056500 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-certs\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056500 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe61fed-819b-45fb-8960-373c4f19988e-istiod-ca-cert\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-data\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe61fed-819b-45fb-8960-373c4f19988e-istio-podinfo\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056726 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-envoy\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056726 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056641    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-socket\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056726 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-istio-token\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.056884 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.056796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts5hr\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-kube-api-access-ts5hr\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.066371 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.066343    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"]
Jul 08 19:35:34.157682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-data\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe61fed-819b-45fb-8960-373c4f19988e-istio-podinfo\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-envoy\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-socket\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-istio-token\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts5hr\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-kube-api-access-ts5hr\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-credential-socket\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-certs\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.157959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.157892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe61fed-819b-45fb-8960-373c4f19988e-istiod-ca-cert\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.158357 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.158177    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-data\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.158357 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.158211    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-socket\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.158357 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.158342    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-credential-socket\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.158518 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.158409    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-certs\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.158589 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.158571    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe61fed-819b-45fb-8960-373c4f19988e-istiod-ca-cert\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.160405 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.160382    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-envoy\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.160557 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.160537    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe61fed-819b-45fb-8960-373c4f19988e-istio-podinfo\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.168176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.168142    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-istio-token\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.168507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.168488    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts5hr\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-kube-api-access-ts5hr\") pod \"e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") " pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.365567 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.365526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:34.522363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.522337    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"]
Jul 08 19:35:34.524561 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:35:34.524529    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe61fed_819b_45fb_8960_373c4f19988e.slice/crio-c873b9a3100730c1303c065b19367b08a54310c2edcb62041c824b700d0dd1c7 WatchSource:0}: Error finding container c873b9a3100730c1303c065b19367b08a54310c2edcb62041c824b700d0dd1c7: Status 404 returned error can't find the container with id c873b9a3100730c1303c065b19367b08a54310c2edcb62041c824b700d0dd1c7
Jul 08 19:35:34.527295 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.527254    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:34.527387 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.527351    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:34.527429 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.527398    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:34.790956 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.790908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" event={"ID":"efe61fed-819b-45fb-8960-373c4f19988e","Type":"ContainerStarted","Data":"8fe2545455bfe259c705a775f84ded90dd48ffa899b273e61821f6252407c895"}
Jul 08 19:35:34.791167 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.791015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" event={"ID":"efe61fed-819b-45fb-8960-373c4f19988e","Type":"ContainerStarted","Data":"c873b9a3100730c1303c065b19367b08a54310c2edcb62041c824b700d0dd1c7"}
Jul 08 19:35:34.825871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:34.825766    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podStartSLOduration=0.825748838 podStartE2EDuration="825.748838ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:34.824212859 +0000 UTC m=+1808.488775411" watchObservedRunningTime="2026-07-08 19:35:34.825748838 +0000 UTC m=+1808.490311389"
Jul 08 19:35:35.366756 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:35.366713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:35.371852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:35.371824    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:35.795799 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:35.795712    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:35.796637 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:35.796618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:36.899614 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.899552    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"]
Jul 08 19:35:36.900128 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.900111    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2873d2d1-6beb-49ad-a7b1-eb9c34ab27ca" containerName="istio-proxy"
Jul 08 19:35:36.902268 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.902250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:36.905670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.905648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3f1b3726-serving-cert\""
Jul 08 19:35:36.905813 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.905686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:35:36.917713 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.917676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"]
Jul 08 19:35:36.988176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.987447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbz5t\" (UniqueName: \"kubernetes.io/projected/fb10a08b-501d-418e-8fa3-af49fe099501-kube-api-access-bbz5t\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:36.988176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.987552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb10a08b-501d-418e-8fa3-af49fe099501-maas-api-tls\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:36.988176 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:36.987642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb10a08b-501d-418e-8fa3-af49fe099501-openshift-service-ca\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.088622 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.088587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbz5t\" (UniqueName: \"kubernetes.io/projected/fb10a08b-501d-418e-8fa3-af49fe099501-kube-api-access-bbz5t\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.088820 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.088653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb10a08b-501d-418e-8fa3-af49fe099501-maas-api-tls\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.088820 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.088698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb10a08b-501d-418e-8fa3-af49fe099501-openshift-service-ca\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.089511 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.089436    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb10a08b-501d-418e-8fa3-af49fe099501-openshift-service-ca\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.091450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.091424    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb10a08b-501d-418e-8fa3-af49fe099501-maas-api-tls\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.098274 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.098247    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbz5t\" (UniqueName: \"kubernetes.io/projected/fb10a08b-501d-418e-8fa3-af49fe099501-kube-api-access-bbz5t\") pod \"maas-api-e2e-mt-3f1b3726-6679754494-k69dj\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.214370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.214261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:37.374012 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.373981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"]
Jul 08 19:35:37.376334 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:35:37.376302    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb10a08b_501d_418e_8fa3_af49fe099501.slice/crio-beca7561e53bb80b996daebed0a8b3da8cbfed429624a55b62de92962247f654 WatchSource:0}: Error finding container beca7561e53bb80b996daebed0a8b3da8cbfed429624a55b62de92962247f654: Status 404 returned error can't find the container with id beca7561e53bb80b996daebed0a8b3da8cbfed429624a55b62de92962247f654
Jul 08 19:35:37.377891 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.377870    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 19:35:37.807027 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:37.806994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj" event={"ID":"fb10a08b-501d-418e-8fa3-af49fe099501","Type":"ContainerStarted","Data":"beca7561e53bb80b996daebed0a8b3da8cbfed429624a55b62de92962247f654"}
Jul 08 19:35:38.813716 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:38.813675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj" event={"ID":"fb10a08b-501d-418e-8fa3-af49fe099501","Type":"ContainerStarted","Data":"e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5"}
Jul 08 19:35:38.814237 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:38.813748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:38.845327 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:38.845273    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj" podStartSLOduration=2.424420502 podStartE2EDuration="2.845256106s" totalImagesPullingTime="420.835604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:38.843794877 +0000 UTC m=+1812.508357427" watchObservedRunningTime="2026-07-08 19:35:38.845256106 +0000 UTC m=+1812.509818657"
Jul 08 19:35:44.826136 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:44.826102    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:35:50.797335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.797294    2563 patch_prober.go:28] interesting pod/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 08 19:35:50.797767 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.797359    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 08 19:35:50.878620 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.878579    2563 generic.go:356] "Generic (PLEG): container finished" podID="efe61fed-819b-45fb-8960-373c4f19988e" containerID="8fe2545455bfe259c705a775f84ded90dd48ffa899b273e61821f6252407c895" exitCode=0
Jul 08 19:35:50.878852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.878617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" event={"ID":"efe61fed-819b-45fb-8960-373c4f19988e","Type":"ContainerDied","Data":"8fe2545455bfe259c705a775f84ded90dd48ffa899b273e61821f6252407c895"}
Jul 08 19:35:50.879393 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.879361    2563 scope.go:117] "RemoveContainer" containerID="8fe2545455bfe259c705a775f84ded90dd48ffa899b273e61821f6252407c895"
Jul 08 19:35:50.880664 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.880632    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:50.880760 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.880714    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:50.880760 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:50.880749    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:35:51.889685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:51.889636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" event={"ID":"efe61fed-819b-45fb-8960-373c4f19988e","Type":"ContainerStarted","Data":"87e5eb1559f44d8ffba4d4bf3b804e7386dbe5bb31c1cc4ba8656ea4df133591"}
Jul 08 19:35:52.366491 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:52.366447    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:52.366740 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:52.366713    2563 patch_prober.go:28] interesting pod/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 08 19:35:52.366820 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:52.366776    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 08 19:35:53.366325 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:53.366281    2563 patch_prober.go:28] interesting pod/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 08 19:35:53.366743 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:53.366348    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 08 19:35:54.366631 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:54.366589    2563 patch_prober.go:28] interesting pod/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 08 19:35:54.367055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:54.366652    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 08 19:35:55.366418 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:55.366376    2563 patch_prober.go:28] interesting pod/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 08 19:35:55.366618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:55.366434    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 08 19:35:56.370751 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:56.370714    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:56.371152 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:56.370971    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:35:56.371749 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:35:56.371730    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:36:14.328527 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.328490    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"]
Jul 08 19:36:14.329124 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.328910    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj" podUID="fb10a08b-501d-418e-8fa3-af49fe099501" containerName="maas-api" containerID="cri-o://e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5" gracePeriod=30
Jul 08 19:36:14.591976 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.591897    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:36:14.699111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.699037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbz5t\" (UniqueName: \"kubernetes.io/projected/fb10a08b-501d-418e-8fa3-af49fe099501-kube-api-access-bbz5t\") pod \"fb10a08b-501d-418e-8fa3-af49fe099501\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") "
Jul 08 19:36:14.699320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.699139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb10a08b-501d-418e-8fa3-af49fe099501-maas-api-tls\") pod \"fb10a08b-501d-418e-8fa3-af49fe099501\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") "
Jul 08 19:36:14.699320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.699181    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb10a08b-501d-418e-8fa3-af49fe099501-openshift-service-ca\") pod \"fb10a08b-501d-418e-8fa3-af49fe099501\" (UID: \"fb10a08b-501d-418e-8fa3-af49fe099501\") "
Jul 08 19:36:14.699573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.699549    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb10a08b-501d-418e-8fa3-af49fe099501-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb10a08b-501d-418e-8fa3-af49fe099501" (UID: "fb10a08b-501d-418e-8fa3-af49fe099501"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:36:14.701380 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.701350    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb10a08b-501d-418e-8fa3-af49fe099501-kube-api-access-bbz5t" (OuterVolumeSpecName: "kube-api-access-bbz5t") pod "fb10a08b-501d-418e-8fa3-af49fe099501" (UID: "fb10a08b-501d-418e-8fa3-af49fe099501"). InnerVolumeSpecName "kube-api-access-bbz5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:36:14.701466 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.701357    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb10a08b-501d-418e-8fa3-af49fe099501-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb10a08b-501d-418e-8fa3-af49fe099501" (UID: "fb10a08b-501d-418e-8fa3-af49fe099501"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:36:14.800771 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.800722    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb10a08b-501d-418e-8fa3-af49fe099501-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:36:14.800771 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.800769    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb10a08b-501d-418e-8fa3-af49fe099501-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:36:14.800990 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:14.800784    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbz5t\" (UniqueName: \"kubernetes.io/projected/fb10a08b-501d-418e-8fa3-af49fe099501-kube-api-access-bbz5t\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:36:15.031261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.031203    2563 generic.go:356] "Generic (PLEG): container finished" podID="fb10a08b-501d-418e-8fa3-af49fe099501" containerID="e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5" exitCode=0
Jul 08 19:36:15.031261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.031250    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"
Jul 08 19:36:15.031261 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.031266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj" event={"ID":"fb10a08b-501d-418e-8fa3-af49fe099501","Type":"ContainerDied","Data":"e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5"}
Jul 08 19:36:15.031568 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.031308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj" event={"ID":"fb10a08b-501d-418e-8fa3-af49fe099501","Type":"ContainerDied","Data":"beca7561e53bb80b996daebed0a8b3da8cbfed429624a55b62de92962247f654"}
Jul 08 19:36:15.031568 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.031330    2563 scope.go:117] "RemoveContainer" containerID="e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5"
Jul 08 19:36:15.041268 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.041246    2563 scope.go:117] "RemoveContainer" containerID="e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5"
Jul 08 19:36:15.041606 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:36:15.041585    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5\": container with ID starting with e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5 not found: ID does not exist" containerID="e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5"
Jul 08 19:36:15.041671 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.041617    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5"} err="failed to get container status \"e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5\": rpc error: code = NotFound desc = could not find container \"e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5\": container with ID starting with e566de7fb012c0cc407f1f0f12335a99f224c94a81717a96fd873a255b7914c5 not found: ID does not exist"
Jul 08 19:36:15.060038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.059995    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"]
Jul 08 19:36:15.065568 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:15.065534    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f1b3726-6679754494-k69dj"]
Jul 08 19:36:16.930922 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:36:16.930886    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb10a08b-501d-418e-8fa3-af49fe099501" path="/var/lib/kubelet/pods/fb10a08b-501d-418e-8fa3-af49fe099501/volumes"
Jul 08 19:37:15.170573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:15.170484    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"]
Jul 08 19:37:15.171160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:15.170700    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy" containerID="cri-o://87e5eb1559f44d8ffba4d4bf3b804e7386dbe5bb31c1cc4ba8656ea4df133591" gracePeriod=30
Jul 08 19:37:20.365831 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.365794    2563 generic.go:356] "Generic (PLEG): container finished" podID="efe61fed-819b-45fb-8960-373c4f19988e" containerID="87e5eb1559f44d8ffba4d4bf3b804e7386dbe5bb31c1cc4ba8656ea4df133591" exitCode=0
Jul 08 19:37:20.366313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.365906    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" event={"ID":"efe61fed-819b-45fb-8960-373c4f19988e","Type":"ContainerDied","Data":"87e5eb1559f44d8ffba4d4bf3b804e7386dbe5bb31c1cc4ba8656ea4df133591"}
Jul 08 19:37:20.366313 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.365972    2563 scope.go:117] "RemoveContainer" containerID="8fe2545455bfe259c705a775f84ded90dd48ffa899b273e61821f6252407c895"
Jul 08 19:37:20.432604 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.432529    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:37:20.478397 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478359    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-istio-token\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478597 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-socket\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478597 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478457    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe61fed-819b-45fb-8960-373c4f19988e-istiod-ca-cert\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478597 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478496    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-envoy\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478597 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478567    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts5hr\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-kube-api-access-ts5hr\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe61fed-819b-45fb-8960-373c4f19988e-istio-podinfo\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-certs\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478666    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-data\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-credential-socket\") pod \"efe61fed-819b-45fb-8960-373c4f19988e\" (UID: \"efe61fed-819b-45fb-8960-373c4f19988e\") "
Jul 08 19:37:20.478836 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.478749    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:37:20.479115 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.479066    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.479320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.479279    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efe61fed-819b-45fb-8960-373c4f19988e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:37:20.479320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.479305    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:37:20.479886 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.479834    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:37:20.480219 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.480179    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-data" (OuterVolumeSpecName: "istio-data") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:37:20.481123 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.481074    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-istio-token" (OuterVolumeSpecName: "istio-token") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:37:20.481966 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.481933    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-kube-api-access-ts5hr" (OuterVolumeSpecName: "kube-api-access-ts5hr") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "kube-api-access-ts5hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:37:20.482242 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.482218    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efe61fed-819b-45fb-8960-373c4f19988e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:37:20.482443 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.482421    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efe61fed-819b-45fb-8960-373c4f19988e" (UID: "efe61fed-819b-45fb-8960-373c4f19988e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:37:20.579760 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579724    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579760 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579757    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579770    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe61fed-819b-45fb-8960-373c4f19988e-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579781    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579790    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts5hr\" (UniqueName: \"kubernetes.io/projected/efe61fed-819b-45fb-8960-373c4f19988e-kube-api-access-ts5hr\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579798    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe61fed-819b-45fb-8960-373c4f19988e-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579806    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:20.579960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:20.579815    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe61fed-819b-45fb-8960-373c4f19988e-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:37:21.373039 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:21.372997    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74" event={"ID":"efe61fed-819b-45fb-8960-373c4f19988e","Type":"ContainerDied","Data":"c873b9a3100730c1303c065b19367b08a54310c2edcb62041c824b700d0dd1c7"}
Jul 08 19:37:21.373039 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:21.373042    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"
Jul 08 19:37:21.373685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:21.373067    2563 scope.go:117] "RemoveContainer" containerID="87e5eb1559f44d8ffba4d4bf3b804e7386dbe5bb31c1cc4ba8656ea4df133591"
Jul 08 19:37:21.397789 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:21.397752    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"]
Jul 08 19:37:21.401495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:21.401462    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3f1b3726-openshift-default-7f974b8f5-7cx74"]
Jul 08 19:37:22.930341 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:37:22.930303    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe61fed-819b-45fb-8960-373c4f19988e" path="/var/lib/kubelet/pods/efe61fed-819b-45fb-8960-373c4f19988e/volumes"
Jul 08 19:38:11.403350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403316    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"]
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403729    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy"
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403741    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy"
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403757    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb10a08b-501d-418e-8fa3-af49fe099501" containerName="maas-api"
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403763    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb10a08b-501d-418e-8fa3-af49fe099501" containerName="maas-api"
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403770    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy"
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403775    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy"
Jul 08 19:38:11.403850 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403848    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy"
Jul 08 19:38:11.404112 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.403858    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb10a08b-501d-418e-8fa3-af49fe099501" containerName="maas-api"
Jul 08 19:38:11.406385 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.406363    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.409267 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.409247    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f953f1-openshift-default-dockercfg-zdz2z\""
Jul 08 19:38:11.420415 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.420386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"]
Jul 08 19:38:11.480033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.479996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-certs\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-credential-socket\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-token\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480239 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzdn9\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-kube-api-access-hzdn9\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480386 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istiod-ca-cert\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480386 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-podinfo\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480468 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-socket\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480468 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-data\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.480564 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.480484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-envoy\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-certs\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581138 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-credential-socket\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-token\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzdn9\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-kube-api-access-hzdn9\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istiod-ca-cert\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581519 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-podinfo\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581519 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-socket\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581624 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-data\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581624 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581545    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-certs\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581624 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581584    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-credential-socket\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581624 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-envoy\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.581917 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.581873    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-socket\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.582038 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.582023    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-data\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.582300 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.582249    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istiod-ca-cert\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.584011 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.583981    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-envoy\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.584293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.584275    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-podinfo\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.591013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.590979    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzdn9\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-kube-api-access-hzdn9\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.591013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.591007    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-token\") pod \"e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") " pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.717406 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.717290    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:11.885178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.885049    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"]
Jul 08 19:38:11.888018 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:38:11.887963    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4187cc3_ca30_4cb1_be48_0580b2dada7c.slice/crio-8245ca6adbd5de0b3dfe8222ed27d771fc554a7590e4293725b47e3f7e5b9ed1 WatchSource:0}: Error finding container 8245ca6adbd5de0b3dfe8222ed27d771fc554a7590e4293725b47e3f7e5b9ed1: Status 404 returned error can't find the container with id 8245ca6adbd5de0b3dfe8222ed27d771fc554a7590e4293725b47e3f7e5b9ed1
Jul 08 19:38:11.890620 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.890581    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:11.890758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.890649    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:11.890758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:11.890688    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:12.644040 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:12.644007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" event={"ID":"e4187cc3-ca30-4cb1-be48-0580b2dada7c","Type":"ContainerStarted","Data":"28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"}
Jul 08 19:38:12.644694 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:12.644059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" event={"ID":"e4187cc3-ca30-4cb1-be48-0580b2dada7c","Type":"ContainerStarted","Data":"8245ca6adbd5de0b3dfe8222ed27d771fc554a7590e4293725b47e3f7e5b9ed1"}
Jul 08 19:38:12.672662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:12.672608    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podStartSLOduration=1.6725915009999999 podStartE2EDuration="1.672591501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:12.669430092 +0000 UTC m=+1966.333992643" watchObservedRunningTime="2026-07-08 19:38:12.672591501 +0000 UTC m=+1966.337154113"
Jul 08 19:38:12.717684 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:12.717639    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:12.722845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:12.722817    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:13.654107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:13.652786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:13.657029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:13.657003    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:14.789131 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.789097    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"]
Jul 08 19:38:14.789635 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.789612    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe61fed-819b-45fb-8960-373c4f19988e" containerName="istio-proxy"
Jul 08 19:38:14.791779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.791759    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:14.794803 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.794777    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:38:14.794939 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.794916    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f953f1-serving-cert\""
Jul 08 19:38:14.808356 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.808325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"]
Jul 08 19:38:14.916364 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.916331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:14.916544 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.916373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c1587c-fc66-4198-b113-3dacb60c446d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:14.916544 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:14.916481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf8pn\" (UniqueName: \"kubernetes.io/projected/60c1587c-fc66-4198-b113-3dacb60c446d-kube-api-access-cf8pn\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.017520 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.017481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.017723 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.017549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c1587c-fc66-4198-b113-3dacb60c446d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.017723 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:38:15.017650    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-serving-cert: secret "maas-api-e2e-inf-a-f953f1-serving-cert" not found
Jul 08 19:38:15.017853 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:38:15.017731    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls podName:60c1587c-fc66-4198-b113-3dacb60c446d nodeName:}" failed. No retries permitted until 2026-07-08 19:38:15.517708742 +0000 UTC m=+1969.182271272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls") pod "maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" (UID: "60c1587c-fc66-4198-b113-3dacb60c446d") : secret "maas-api-e2e-inf-a-f953f1-serving-cert" not found
Jul 08 19:38:15.018454 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.018416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf8pn\" (UniqueName: \"kubernetes.io/projected/60c1587c-fc66-4198-b113-3dacb60c446d-kube-api-access-cf8pn\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.018589 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.018523    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c1587c-fc66-4198-b113-3dacb60c446d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.032870 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.032834    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf8pn\" (UniqueName: \"kubernetes.io/projected/60c1587c-fc66-4198-b113-3dacb60c446d-kube-api-access-cf8pn\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.523762 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.523720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.526387 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.526368    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls\") pod \"maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.703310 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.703272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:15.847957 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:15.847925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"]
Jul 08 19:38:15.851133 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:38:15.851073    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c1587c_fc66_4198_b113_3dacb60c446d.slice/crio-8948c297afb8e143c91d6129c031cce6168a1bc189ebfeb0394aa2cf5e435439 WatchSource:0}: Error finding container 8948c297afb8e143c91d6129c031cce6168a1bc189ebfeb0394aa2cf5e435439: Status 404 returned error can't find the container with id 8948c297afb8e143c91d6129c031cce6168a1bc189ebfeb0394aa2cf5e435439
Jul 08 19:38:16.669465 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:16.669425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" event={"ID":"60c1587c-fc66-4198-b113-3dacb60c446d","Type":"ContainerStarted","Data":"09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb"}
Jul 08 19:38:16.669682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:16.669496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" event={"ID":"60c1587c-fc66-4198-b113-3dacb60c446d","Type":"ContainerStarted","Data":"8948c297afb8e143c91d6129c031cce6168a1bc189ebfeb0394aa2cf5e435439"}
Jul 08 19:38:16.669682 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:16.669613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:16.691738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:16.691683    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" podStartSLOduration=2.189651547 podStartE2EDuration="2.691664192s" totalImagesPullingTime="502.012645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:16.689864755 +0000 UTC m=+1970.354427307" watchObservedRunningTime="2026-07-08 19:38:16.691664192 +0000 UTC m=+1970.356226775"
Jul 08 19:38:19.940821 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.940773    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"]
Jul 08 19:38:19.944376 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.944346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.948687 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.948660    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a246eb-openshift-default-dockercfg-9pndr\""
Jul 08 19:38:19.963643 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.963610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"]
Jul 08 19:38:19.969671 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-credential-socket\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.969833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-data\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.969833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-podinfo\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.969833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-envoy\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.969833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-socket\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.970117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n6xq\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-kube-api-access-7n6xq\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.970117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-certs\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.970117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-token\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:19.970117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:19.969900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb8260c-5b0b-4a48-aea4-501901b7380e-istiod-ca-cert\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070443 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-data\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-podinfo\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-envoy\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-socket\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n6xq\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-kube-api-access-7n6xq\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-certs\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.070640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-token\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.071011 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb8260c-5b0b-4a48-aea4-501901b7380e-istiod-ca-cert\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.071011 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-credential-socket\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.071011 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070886    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-data\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.071011 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.070978    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-certs\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.071667 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.071545    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb8260c-5b0b-4a48-aea4-501901b7380e-istiod-ca-cert\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.071808 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.071783    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-socket\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.072004 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.071975    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-credential-socket\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.073335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.073317    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-envoy\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.073548 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.073528    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-podinfo\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.079759 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.079735    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-token\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.079897 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.079879    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n6xq\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-kube-api-access-7n6xq\") pod \"e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") " pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.255425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.255325    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:20.408008 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.407980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"]
Jul 08 19:38:20.411645 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:38:20.411601    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb8260c_5b0b_4a48_aea4_501901b7380e.slice/crio-1288facc0cca7f856b8077fc2e0f12162d5f3e858ecc1e245cf49a96421a7be6 WatchSource:0}: Error finding container 1288facc0cca7f856b8077fc2e0f12162d5f3e858ecc1e245cf49a96421a7be6: Status 404 returned error can't find the container with id 1288facc0cca7f856b8077fc2e0f12162d5f3e858ecc1e245cf49a96421a7be6
Jul 08 19:38:20.414166 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.414127    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:20.414262 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.414222    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:20.414339 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.414301    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:20.698460 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.698424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" event={"ID":"5cb8260c-5b0b-4a48-aea4-501901b7380e","Type":"ContainerStarted","Data":"06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"}
Jul 08 19:38:20.698670 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.698487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" event={"ID":"5cb8260c-5b0b-4a48-aea4-501901b7380e","Type":"ContainerStarted","Data":"1288facc0cca7f856b8077fc2e0f12162d5f3e858ecc1e245cf49a96421a7be6"}
Jul 08 19:38:20.725528 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:20.725269    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podStartSLOduration=1.725244362 podStartE2EDuration="1.725244362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:20.723555948 +0000 UTC m=+1974.388118523" watchObservedRunningTime="2026-07-08 19:38:20.725244362 +0000 UTC m=+1974.389806916"
Jul 08 19:38:21.255808 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:21.255767    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:21.260535 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:21.260508    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:21.703828 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:21.703795    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:21.704746 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:21.704723    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:22.682728 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.682686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:38:22.840455 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.840398    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"]
Jul 08 19:38:22.846596 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.845638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:22.848926 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.848349    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a246eb-serving-cert\""
Jul 08 19:38:22.853355 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.853327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"]
Jul 08 19:38:22.900929 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.900896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q49v\" (UniqueName: \"kubernetes.io/projected/7897e05e-05e7-4f84-9ad8-0bf469895582-kube-api-access-2q49v\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:22.901178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.900955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7897e05e-05e7-4f84-9ad8-0bf469895582-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:22.901178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:22.901029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.002106 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.001981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q49v\" (UniqueName: \"kubernetes.io/projected/7897e05e-05e7-4f84-9ad8-0bf469895582-kube-api-access-2q49v\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.002426 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.002333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7897e05e-05e7-4f84-9ad8-0bf469895582-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.002557 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.002533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.002621 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:38:23.002612    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-serving-cert: secret "maas-api-e2e-inf-b-a246eb-serving-cert" not found
Jul 08 19:38:23.002690 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:38:23.002682    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls podName:7897e05e-05e7-4f84-9ad8-0bf469895582 nodeName:}" failed. No retries permitted until 2026-07-08 19:38:23.502661515 +0000 UTC m=+1977.167224043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls") pod "maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" (UID: "7897e05e-05e7-4f84-9ad8-0bf469895582") : secret "maas-api-e2e-inf-b-a246eb-serving-cert" not found
Jul 08 19:38:23.003331 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.003302    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7897e05e-05e7-4f84-9ad8-0bf469895582-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.014884 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.014860    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q49v\" (UniqueName: \"kubernetes.io/projected/7897e05e-05e7-4f84-9ad8-0bf469895582-kube-api-access-2q49v\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.508437 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.508397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.511070 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.511041    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls\") pod \"maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.761258 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.761162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:23.924840 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:23.924808    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"]
Jul 08 19:38:23.926242 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:38:23.926218    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7897e05e_05e7_4f84_9ad8_0bf469895582.slice/crio-564a92af06c1d68b982f75c1e30a3cd12e8a4dcb443e60a9001c27a309ffc356 WatchSource:0}: Error finding container 564a92af06c1d68b982f75c1e30a3cd12e8a4dcb443e60a9001c27a309ffc356: Status 404 returned error can't find the container with id 564a92af06c1d68b982f75c1e30a3cd12e8a4dcb443e60a9001c27a309ffc356
Jul 08 19:38:24.721585 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:24.721545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" event={"ID":"7897e05e-05e7-4f84-9ad8-0bf469895582","Type":"ContainerStarted","Data":"29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec"}
Jul 08 19:38:24.721763 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:24.721600    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" event={"ID":"7897e05e-05e7-4f84-9ad8-0bf469895582","Type":"ContainerStarted","Data":"564a92af06c1d68b982f75c1e30a3cd12e8a4dcb443e60a9001c27a309ffc356"}
Jul 08 19:38:24.722187 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:24.722170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:24.746581 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:24.746497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" podStartSLOduration=2.319701528 podStartE2EDuration="2.746464889s" totalImagesPullingTime="426.763361ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:24.746315116 +0000 UTC m=+1978.410877666" watchObservedRunningTime="2026-07-08 19:38:24.746464889 +0000 UTC m=+1978.411027440"
Jul 08 19:38:29.694358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:29.694296    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 19:38:29.694820 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:29.694398    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 19:38:31.737530 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.737491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:38:31.761919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.761885    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerID="28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037" exitCode=0
Jul 08 19:38:31.762106 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.761961    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" event={"ID":"e4187cc3-ca30-4cb1-be48-0580b2dada7c","Type":"ContainerDied","Data":"28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"}
Jul 08 19:38:31.762401 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.762388    2563 scope.go:117] "RemoveContainer" containerID="28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"
Jul 08 19:38:31.763438 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.763413    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:31.763495 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.763475    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:31.763542 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:31.763509    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:32.768584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:32.768541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" event={"ID":"e4187cc3-ca30-4cb1-be48-0580b2dada7c","Type":"ContainerStarted","Data":"e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1"}
Jul 08 19:38:33.717987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:33.717946    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:33.718402 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:33.718359    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 08 19:38:33.718657 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:33.718611    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 08 19:38:34.718095 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:34.718030    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 08 19:38:34.718634 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:34.718142    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 08 19:38:35.718861 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:35.718808    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 08 19:38:35.719406 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:35.718902    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 08 19:38:36.764961 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:36.764876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:36.765356 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:36.765180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:36.766191 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:36.766172    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:38:37.745292 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:37.745241    2563 patch_prober.go:28] interesting pod/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 19:38:37.745484 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:37.745318    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 19:38:39.843327 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:39.843221    2563 generic.go:356] "Generic (PLEG): container finished" podID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerID="06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433" exitCode=0
Jul 08 19:38:39.843327 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:39.843301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" event={"ID":"5cb8260c-5b0b-4a48-aea4-501901b7380e","Type":"ContainerDied","Data":"06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"}
Jul 08 19:38:39.843825 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:39.843802    2563 scope.go:117] "RemoveContainer" containerID="06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"
Jul 08 19:38:39.844899 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:39.844864    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:39.845019 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:39.844954    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:39.845019 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:39.845004    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:38:40.850360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:40.850324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" event={"ID":"5cb8260c-5b0b-4a48-aea4-501901b7380e","Type":"ContainerStarted","Data":"7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c"}
Jul 08 19:38:41.256588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:41.256538    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:41.256837 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:41.256816    2563 patch_prober.go:28] interesting pod/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 19:38:41.256887 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:41.256865    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 19:38:42.256853 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:42.256808    2563 patch_prober.go:28] interesting pod/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 19:38:42.257467 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:42.256874    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 19:38:43.256070 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:43.256029    2563 patch_prober.go:28] interesting pod/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 19:38:43.256283 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:43.256141    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 19:38:44.255812 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:44.255763    2563 patch_prober.go:28] interesting pod/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 19:38:44.256215 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:44.255834    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 19:38:45.259541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:45.259504    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:45.259957 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:45.259897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:38:45.260852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:38:45.260824    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:39:53.463562 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.463476    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"]
Jul 08 19:39:53.464122 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.463719    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" podUID="60c1587c-fc66-4198-b113-3dacb60c446d" containerName="maas-api" containerID="cri-o://09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb" gracePeriod=30
Jul 08 19:39:53.732627 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.732590    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:39:53.860030 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.859993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c1587c-fc66-4198-b113-3dacb60c446d-openshift-service-ca\") pod \"60c1587c-fc66-4198-b113-3dacb60c446d\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") "
Jul 08 19:39:53.860245 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.860054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf8pn\" (UniqueName: \"kubernetes.io/projected/60c1587c-fc66-4198-b113-3dacb60c446d-kube-api-access-cf8pn\") pod \"60c1587c-fc66-4198-b113-3dacb60c446d\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") "
Jul 08 19:39:53.860320 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.860263    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls\") pod \"60c1587c-fc66-4198-b113-3dacb60c446d\" (UID: \"60c1587c-fc66-4198-b113-3dacb60c446d\") "
Jul 08 19:39:53.860441 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.860417    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60c1587c-fc66-4198-b113-3dacb60c446d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60c1587c-fc66-4198-b113-3dacb60c446d" (UID: "60c1587c-fc66-4198-b113-3dacb60c446d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:39:53.860667 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.860647    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c1587c-fc66-4198-b113-3dacb60c446d-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:39:53.862357 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.862333    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60c1587c-fc66-4198-b113-3dacb60c446d" (UID: "60c1587c-fc66-4198-b113-3dacb60c446d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:39:53.862454 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.862387    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60c1587c-fc66-4198-b113-3dacb60c446d-kube-api-access-cf8pn" (OuterVolumeSpecName: "kube-api-access-cf8pn") pod "60c1587c-fc66-4198-b113-3dacb60c446d" (UID: "60c1587c-fc66-4198-b113-3dacb60c446d"). InnerVolumeSpecName "kube-api-access-cf8pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:39:53.962192 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.962150    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf8pn\" (UniqueName: \"kubernetes.io/projected/60c1587c-fc66-4198-b113-3dacb60c446d-kube-api-access-cf8pn\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:39:53.962192 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:53.962185    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c1587c-fc66-4198-b113-3dacb60c446d-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:39:54.231371 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.231332    2563 generic.go:356] "Generic (PLEG): container finished" podID="60c1587c-fc66-4198-b113-3dacb60c446d" containerID="09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb" exitCode=0
Jul 08 19:39:54.231371 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.231372    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"
Jul 08 19:39:54.231606 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.231416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" event={"ID":"60c1587c-fc66-4198-b113-3dacb60c446d","Type":"ContainerDied","Data":"09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb"}
Jul 08 19:39:54.231606 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.231477    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j" event={"ID":"60c1587c-fc66-4198-b113-3dacb60c446d","Type":"ContainerDied","Data":"8948c297afb8e143c91d6129c031cce6168a1bc189ebfeb0394aa2cf5e435439"}
Jul 08 19:39:54.231606 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.231505    2563 scope.go:117] "RemoveContainer" containerID="09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb"
Jul 08 19:39:54.241458 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.241439    2563 scope.go:117] "RemoveContainer" containerID="09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb"
Jul 08 19:39:54.241775 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:39:54.241758    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb\": container with ID starting with 09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb not found: ID does not exist" containerID="09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb"
Jul 08 19:39:54.241824 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.241786    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb"} err="failed to get container status \"09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb\": rpc error: code = NotFound desc = could not find container \"09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb\": container with ID starting with 09b83ff6e37e4d63b3496494257c0098346070bf2ba463ecb71485a26dfd2ddb not found: ID does not exist"
Jul 08 19:39:54.254508 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.254480    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"]
Jul 08 19:39:54.258438 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.258410    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f953f1-66dd8b4b58-54b2j"]
Jul 08 19:39:54.939137 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:39:54.939064    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60c1587c-fc66-4198-b113-3dacb60c446d" path="/var/lib/kubelet/pods/60c1587c-fc66-4198-b113-3dacb60c446d/volumes"
Jul 08 19:40:35.396586 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:35.396540    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"]
Jul 08 19:40:35.397155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:35.396832    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy" containerID="cri-o://e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1" gracePeriod=30
Jul 08 19:40:36.623571 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.623530    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"]
Jul 08 19:40:36.624152 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.623773    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" podUID="7897e05e-05e7-4f84-9ad8-0bf469895582" containerName="maas-api" containerID="cri-o://29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec" gracePeriod=30
Jul 08 19:40:36.734151 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.733814    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" podUID="7897e05e-05e7-4f84-9ad8-0bf469895582" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.59:8443/health\": dial tcp 10.134.0.59:8443: connect: connection refused"
Jul 08 19:40:36.767867 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.767816    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 19:40:36.768049 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.767902    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 19:40:36.883445 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.883373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:40:36.915998 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.915963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls\") pod \"7897e05e-05e7-4f84-9ad8-0bf469895582\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") "
Jul 08 19:40:36.916213 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.916048    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7897e05e-05e7-4f84-9ad8-0bf469895582-openshift-service-ca\") pod \"7897e05e-05e7-4f84-9ad8-0bf469895582\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") "
Jul 08 19:40:36.916213 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.916153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q49v\" (UniqueName: \"kubernetes.io/projected/7897e05e-05e7-4f84-9ad8-0bf469895582-kube-api-access-2q49v\") pod \"7897e05e-05e7-4f84-9ad8-0bf469895582\" (UID: \"7897e05e-05e7-4f84-9ad8-0bf469895582\") "
Jul 08 19:40:36.916389 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.916364    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7897e05e-05e7-4f84-9ad8-0bf469895582-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7897e05e-05e7-4f84-9ad8-0bf469895582" (UID: "7897e05e-05e7-4f84-9ad8-0bf469895582"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:40:36.916516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.916500    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7897e05e-05e7-4f84-9ad8-0bf469895582-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:36.918409 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.918386    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7897e05e-05e7-4f84-9ad8-0bf469895582-kube-api-access-2q49v" (OuterVolumeSpecName: "kube-api-access-2q49v") pod "7897e05e-05e7-4f84-9ad8-0bf469895582" (UID: "7897e05e-05e7-4f84-9ad8-0bf469895582"). InnerVolumeSpecName "kube-api-access-2q49v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:40:36.918509 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:36.918494    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7897e05e-05e7-4f84-9ad8-0bf469895582" (UID: "7897e05e-05e7-4f84-9ad8-0bf469895582"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:40:37.017361 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.017325    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q49v\" (UniqueName: \"kubernetes.io/projected/7897e05e-05e7-4f84-9ad8-0bf469895582-kube-api-access-2q49v\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:37.017361 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.017356    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7897e05e-05e7-4f84-9ad8-0bf469895582-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:37.458284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.458245    2563 generic.go:356] "Generic (PLEG): container finished" podID="7897e05e-05e7-4f84-9ad8-0bf469895582" containerID="29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec" exitCode=0
Jul 08 19:40:37.458481 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.458307    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"
Jul 08 19:40:37.458481 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.458321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" event={"ID":"7897e05e-05e7-4f84-9ad8-0bf469895582","Type":"ContainerDied","Data":"29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec"}
Jul 08 19:40:37.458481 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.458396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm" event={"ID":"7897e05e-05e7-4f84-9ad8-0bf469895582","Type":"ContainerDied","Data":"564a92af06c1d68b982f75c1e30a3cd12e8a4dcb443e60a9001c27a309ffc356"}
Jul 08 19:40:37.458481 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.458420    2563 scope.go:117] "RemoveContainer" containerID="29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec"
Jul 08 19:40:37.467696 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.467676    2563 scope.go:117] "RemoveContainer" containerID="29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec"
Jul 08 19:40:37.467955 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:37.467933    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec\": container with ID starting with 29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec not found: ID does not exist" containerID="29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec"
Jul 08 19:40:37.468047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.467967    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec"} err="failed to get container status \"29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec\": rpc error: code = NotFound desc = could not find container \"29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec\": container with ID starting with 29384633737a69f033d3fe69cdd704d93dfa331c3e6f3f36c9a6fb254d3ae2ec not found: ID does not exist"
Jul 08 19:40:37.476055 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.476027    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"]
Jul 08 19:40:37.478205 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:37.478183    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a246eb-84fc5b4b85-r65gm"]
Jul 08 19:40:38.930552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:38.930518    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7897e05e-05e7-4f84-9ad8-0bf469895582" path="/var/lib/kubelet/pods/7897e05e-05e7-4f84-9ad8-0bf469895582/volumes"
Jul 08 19:40:40.668013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.667944    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:40:40.754116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754071    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-envoy\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-token\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-podinfo\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-credential-socket\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754236    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzdn9\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-kube-api-access-hzdn9\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754261    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-data\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754305 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754293    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istiod-ca-cert\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754610 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754319    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-certs\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754610 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-socket\") pod \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\" (UID: \"e4187cc3-ca30-4cb1-be48-0580b2dada7c\") "
Jul 08 19:40:40.754825 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754763    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:40:40.754999 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.754853    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:40.754999 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.755017    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:40.755408 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.755378    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:40.755502 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.755457    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:40.756646 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.756608    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-kube-api-access-hzdn9" (OuterVolumeSpecName: "kube-api-access-hzdn9") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "kube-api-access-hzdn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:40:40.756646 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.756627    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:40:40.756950 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.756931    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:40.756950 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.756939    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4187cc3-ca30-4cb1-be48-0580b2dada7c" (UID: "e4187cc3-ca30-4cb1-be48-0580b2dada7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:40:40.855507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855472    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855500    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855507 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855513    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855522    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855531    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855540    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855547    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855556    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzdn9\" (UniqueName: \"kubernetes.io/projected/e4187cc3-ca30-4cb1-be48-0580b2dada7c-kube-api-access-hzdn9\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:40.855750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:40.855565    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4187cc3-ca30-4cb1-be48-0580b2dada7c-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:41.481904 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.481869    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerID="e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1" exitCode=0
Jul 08 19:40:41.481904 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.481908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" event={"ID":"e4187cc3-ca30-4cb1-be48-0580b2dada7c","Type":"ContainerDied","Data":"e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1"}
Jul 08 19:40:41.482177 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.481931    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"
Jul 08 19:40:41.482177 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.481950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn" event={"ID":"e4187cc3-ca30-4cb1-be48-0580b2dada7c","Type":"ContainerDied","Data":"8245ca6adbd5de0b3dfe8222ed27d771fc554a7590e4293725b47e3f7e5b9ed1"}
Jul 08 19:40:41.482177 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.481972    2563 scope.go:117] "RemoveContainer" containerID="e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1"
Jul 08 19:40:41.491658 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.491634    2563 scope.go:117] "RemoveContainer" containerID="28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"
Jul 08 19:40:41.502920 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.502882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"]
Jul 08 19:40:41.503554 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.503535    2563 scope.go:117] "RemoveContainer" containerID="e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1"
Jul 08 19:40:41.503853 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:41.503827    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1\": container with ID starting with e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1 not found: ID does not exist" containerID="e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1"
Jul 08 19:40:41.503967 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.503863    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1"} err="failed to get container status \"e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1\": rpc error: code = NotFound desc = could not find container \"e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1\": container with ID starting with e0bff4c785016b336c8207924788371612f5ec65e28f7ce110e6de86877c52e1 not found: ID does not exist"
Jul 08 19:40:41.503967 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.503900    2563 scope.go:117] "RemoveContainer" containerID="28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"
Jul 08 19:40:41.504166 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:41.504147    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037\": container with ID starting with 28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037 not found: ID does not exist" containerID="28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"
Jul 08 19:40:41.504236 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.504178    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037"} err="failed to get container status \"28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037\": rpc error: code = NotFound desc = could not find container \"28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037\": container with ID starting with 28b3d936a78fe423879cc55dc3b01712d05a64e00ef3e64c7b9a7977d9df3037 not found: ID does not exist"
Jul 08 19:40:41.504857 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:41.504840    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f953f1-openshift-default-66dcf48db5-b6jvn"]
Jul 08 19:40:42.479491 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:42.479449    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"]
Jul 08 19:40:42.479977 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:42.479741    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" containerID="cri-o://7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c" gracePeriod=30
Jul 08 19:40:42.930837 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:42.930798    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" path="/var/lib/kubelet/pods/e4187cc3-ca30-4cb1-be48-0580b2dada7c/volumes"
Jul 08 19:40:43.350813 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.350679    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"]
Jul 08 19:40:43.351330 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351308    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7897e05e-05e7-4f84-9ad8-0bf469895582" containerName="maas-api"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351334    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7897e05e-05e7-4f84-9ad8-0bf469895582" containerName="maas-api"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351350    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351359    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351383    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60c1587c-fc66-4198-b113-3dacb60c446d" containerName="maas-api"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351392    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="60c1587c-fc66-4198-b113-3dacb60c446d" containerName="maas-api"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351407    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy"
Jul 08 19:40:43.351413 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351414    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy"
Jul 08 19:40:43.351629 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351508    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy"
Jul 08 19:40:43.351629 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351518    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7897e05e-05e7-4f84-9ad8-0bf469895582" containerName="maas-api"
Jul 08 19:40:43.351629 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.351528    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="60c1587c-fc66-4198-b113-3dacb60c446d" containerName="maas-api"
Jul 08 19:40:43.356377 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.356354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.359061 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.359038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8627fc-openshift-default-dockercfg-p4qlf\""
Jul 08 19:40:43.368811 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.368781    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"]
Jul 08 19:40:43.483189 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-certs\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483189 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-data\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-socket\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-token\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-credential-socket\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed41d798-c5e3-477c-bcee-dc541c214b0b-istiod-ca-cert\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-podinfo\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsrtf\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-kube-api-access-hsrtf\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.483691 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.483530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-envoy\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.584831 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.584795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed41d798-c5e3-477c-bcee-dc541c214b0b-istiod-ca-cert\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585018 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.584851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-podinfo\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585018 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.584882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsrtf\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-kube-api-access-hsrtf\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585018 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.584914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-envoy\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585018 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.584985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-certs\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585018 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-data\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585312 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-socket\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585312 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-token\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585312 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-credential-socket\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585467 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585383    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-certs\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585520 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585490    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-socket\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585548    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-data\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.585929 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.585908    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-credential-socket\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.586273 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.586251    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed41d798-c5e3-477c-bcee-dc541c214b0b-istiod-ca-cert\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.587529 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.587506    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-envoy\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.587707 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.587686    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-podinfo\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.594856 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.594831    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-token\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.595105 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.595059    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsrtf\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-kube-api-access-hsrtf\") pod \"e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") " pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.666584 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.666549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:43.817377 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.817352    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"]
Jul 08 19:40:43.819515 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:40:43.819482    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded41d798_c5e3_477c_bcee_dc541c214b0b.slice/crio-9608f760e9239cfcfd32d61e86bfa71440bb5dcf0fb670d6eb7605df13ff10aa WatchSource:0}: Error finding container 9608f760e9239cfcfd32d61e86bfa71440bb5dcf0fb670d6eb7605df13ff10aa: Status 404 returned error can't find the container with id 9608f760e9239cfcfd32d61e86bfa71440bb5dcf0fb670d6eb7605df13ff10aa
Jul 08 19:40:43.821731 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.821703    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:40:43.821790 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.821768    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:40:43.821821 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:43.821797    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:40:44.503628 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:44.503587    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" event={"ID":"ed41d798-c5e3-477c-bcee-dc541c214b0b","Type":"ContainerStarted","Data":"7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"}
Jul 08 19:40:44.504132 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:44.503655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" event={"ID":"ed41d798-c5e3-477c-bcee-dc541c214b0b","Type":"ContainerStarted","Data":"9608f760e9239cfcfd32d61e86bfa71440bb5dcf0fb670d6eb7605df13ff10aa"}
Jul 08 19:40:44.525711 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:44.525655    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podStartSLOduration=1.5256377639999998 podStartE2EDuration="1.525637764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:40:44.524803766 +0000 UTC m=+2118.189366320" watchObservedRunningTime="2026-07-08 19:40:44.525637764 +0000 UTC m=+2118.190200313"
Jul 08 19:40:44.667749 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:44.667714    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:44.669411 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:44.669386    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 19:40:44.669506 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:44.669443    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 19:40:45.262544 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:45.262502    2563 patch_prober.go:28] interesting pod/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 19:40:45.262782 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:45.262571    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 19:40:45.672784 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:45.672737    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:46.515426 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.515385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:46.516485 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.516465    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:40:46.541827 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.541793    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"]
Jul 08 19:40:46.542516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.542498    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4187cc3-ca30-4cb1-be48-0580b2dada7c" containerName="istio-proxy"
Jul 08 19:40:46.545910 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.545886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.549061 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.549029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8627fc-serving-cert\""
Jul 08 19:40:46.549363 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.549342    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:40:46.570801 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.570763    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"]
Jul 08 19:40:46.619552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.619523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4tfd\" (UniqueName: \"kubernetes.io/projected/9b9ae576-6037-4f10-99eb-ac53d222ef59-kube-api-access-j4tfd\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.619741 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.619570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9ae576-6037-4f10-99eb-ac53d222ef59-openshift-service-ca\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.619741 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.619685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b9ae576-6037-4f10-99eb-ac53d222ef59-maas-api-tls\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.720544 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.720504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4tfd\" (UniqueName: \"kubernetes.io/projected/9b9ae576-6037-4f10-99eb-ac53d222ef59-kube-api-access-j4tfd\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.720990 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.720556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9ae576-6037-4f10-99eb-ac53d222ef59-openshift-service-ca\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.720990 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.720622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b9ae576-6037-4f10-99eb-ac53d222ef59-maas-api-tls\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.721274 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.721255    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9ae576-6037-4f10-99eb-ac53d222ef59-openshift-service-ca\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.723358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.723327    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b9ae576-6037-4f10-99eb-ac53d222ef59-maas-api-tls\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.732760 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.732726    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4tfd\" (UniqueName: \"kubernetes.io/projected/9b9ae576-6037-4f10-99eb-ac53d222ef59-kube-api-access-j4tfd\") pod \"maas-api-e2e-api-a-8627fc-64bf954c-nvx9m\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:46.868001 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:46.867965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:47.022178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.022150    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"]
Jul 08 19:40:47.024472 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:40:47.024430    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b9ae576_6037_4f10_99eb_ac53d222ef59.slice/crio-21ecf3bee6e8be80717aca300f4ee72e6acd5392fe90566f417a84a0994976b3 WatchSource:0}: Error finding container 21ecf3bee6e8be80717aca300f4ee72e6acd5392fe90566f417a84a0994976b3: Status 404 returned error can't find the container with id 21ecf3bee6e8be80717aca300f4ee72e6acd5392fe90566f417a84a0994976b3
Jul 08 19:40:47.026300 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.026278    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 19:40:47.522398 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.522359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m" event={"ID":"9b9ae576-6037-4f10-99eb-ac53d222ef59","Type":"ContainerStarted","Data":"21ecf3bee6e8be80717aca300f4ee72e6acd5392fe90566f417a84a0994976b3"}
Jul 08 19:40:47.750312 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.750245    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:40:47.832832 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832794    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-token\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-socket\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-certs\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832933    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-envoy\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832955    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb8260c-5b0b-4a48-aea4-501901b7380e-istiod-ca-cert\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832975    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n6xq\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-kube-api-access-7n6xq\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.832999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-credential-socket\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833344 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833043    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-data\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833344 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833070    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-podinfo\") pod \"5cb8260c-5b0b-4a48-aea4-501901b7380e\" (UID: \"5cb8260c-5b0b-4a48-aea4-501901b7380e\") "
Jul 08 19:40:47.833344 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833238    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:47.833344 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833289    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:47.833560 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833398    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb8260c-5b0b-4a48-aea4-501901b7380e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:40:47.833609 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833556    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.833609 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833594    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb8260c-5b0b-4a48-aea4-501901b7380e-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.833748 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.833608    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.834198 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.834164    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:47.834350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.834256    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:47.835521 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.835496    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:40:47.835521 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.835512    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:40:47.835661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.835530    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:40:47.835661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.835530    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-kube-api-access-7n6xq" (OuterVolumeSpecName: "kube-api-access-7n6xq") pod "5cb8260c-5b0b-4a48-aea4-501901b7380e" (UID: "5cb8260c-5b0b-4a48-aea4-501901b7380e"). InnerVolumeSpecName "kube-api-access-7n6xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:40:47.934435 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.934391    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.934435 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.934426    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n6xq\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-kube-api-access-7n6xq\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.934435 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.934440    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.934697 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.934451    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.934697 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.934462    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:47.934697 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:47.934472    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb8260c-5b0b-4a48-aea4-501901b7380e-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:40:48.529632 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.529600    2563 generic.go:356] "Generic (PLEG): container finished" podID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerID="7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c" exitCode=0
Jul 08 19:40:48.529840 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.529646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" event={"ID":"5cb8260c-5b0b-4a48-aea4-501901b7380e","Type":"ContainerDied","Data":"7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c"}
Jul 08 19:40:48.529840 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.529657    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"
Jul 08 19:40:48.529840 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.529707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx" event={"ID":"5cb8260c-5b0b-4a48-aea4-501901b7380e","Type":"ContainerDied","Data":"1288facc0cca7f856b8077fc2e0f12162d5f3e858ecc1e245cf49a96421a7be6"}
Jul 08 19:40:48.529840 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.529729    2563 scope.go:117] "RemoveContainer" containerID="7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c"
Jul 08 19:40:48.531618 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.531592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m" event={"ID":"9b9ae576-6037-4f10-99eb-ac53d222ef59","Type":"ContainerStarted","Data":"6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377"}
Jul 08 19:40:48.531763 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.531664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:48.539419 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.539395    2563 scope.go:117] "RemoveContainer" containerID="06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"
Jul 08 19:40:48.548139 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.548115    2563 scope.go:117] "RemoveContainer" containerID="7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c"
Jul 08 19:40:48.548429 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:48.548410    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c\": container with ID starting with 7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c not found: ID does not exist" containerID="7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c"
Jul 08 19:40:48.548497 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.548438    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c"} err="failed to get container status \"7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c\": rpc error: code = NotFound desc = could not find container \"7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c\": container with ID starting with 7fa2398ee7ebaadc388d690f882a2e383ee6d4f199b0bb329f18ea267e09172c not found: ID does not exist"
Jul 08 19:40:48.548497 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.548457    2563 scope.go:117] "RemoveContainer" containerID="06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"
Jul 08 19:40:48.548665 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:48.548647    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433\": container with ID starting with 06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433 not found: ID does not exist" containerID="06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"
Jul 08 19:40:48.548709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.548671    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433"} err="failed to get container status \"06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433\": rpc error: code = NotFound desc = could not find container \"06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433\": container with ID starting with 06359691b63f30122179d428404013ac4584aca95af5265e7a2174880cb29433 not found: ID does not exist"
Jul 08 19:40:48.555523 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.555476    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m" podStartSLOduration=2.078751846 podStartE2EDuration="2.555460845s" totalImagesPullingTime="476.708999ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:40:48.553799783 +0000 UTC m=+2122.218362333" watchObservedRunningTime="2026-07-08 19:40:48.555460845 +0000 UTC m=+2122.220023395"
Jul 08 19:40:48.569346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.569313    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"]
Jul 08 19:40:48.573198 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.573171    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a246eb-openshift-default-6776c5c86c-wrmdx"]
Jul 08 19:40:48.931240 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:48.931201    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" path="/var/lib/kubelet/pods/5cb8260c-5b0b-4a48-aea4-501901b7380e/volumes"
Jul 08 19:40:51.713236 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.713196    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"]
Jul 08 19:40:51.713841 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.713819    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy"
Jul 08 19:40:51.713936 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.713845    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy"
Jul 08 19:40:51.713936 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.713858    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy"
Jul 08 19:40:51.713936 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.713866    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy"
Jul 08 19:40:51.714105 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.713970    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy"
Jul 08 19:40:51.718509 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.718486    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.721052 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.721028    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e62be2-openshift-default-dockercfg-vjnlr\""
Jul 08 19:40:51.731755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.731726    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"]
Jul 08 19:40:51.773387 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-data\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773387 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-istio-token\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0dec084-f070-4a16-b762-759e9326afaa-istiod-ca-cert\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nj9k\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-kube-api-access-4nj9k\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-credential-socket\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-socket\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-envoy\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.773639 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773605    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0dec084-f070-4a16-b762-759e9326afaa-istio-podinfo\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.774015 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.773694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-certs\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.874883 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.874849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-data\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.874883 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.874885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-istio-token\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875162 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.874917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0dec084-f070-4a16-b762-759e9326afaa-istiod-ca-cert\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875162 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.874957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nj9k\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-kube-api-access-4nj9k\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875162 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.874985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-credential-socket\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875362 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875335    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-data\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875362 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875345    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-credential-socket\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875567 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-socket\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-envoy\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0dec084-f070-4a16-b762-759e9326afaa-istio-podinfo\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875811 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875719    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0dec084-f070-4a16-b762-759e9326afaa-istiod-ca-cert\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875811 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-certs\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.875926 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.875886    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-socket\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.876115 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.876074    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-certs\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.878445 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.878417    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-envoy\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.879111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.879071    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0dec084-f070-4a16-b762-759e9326afaa-istio-podinfo\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.887782 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.887752    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nj9k\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-kube-api-access-4nj9k\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:51.887998 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:51.887832    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-istio-token\") pod \"e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") " pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:52.033442 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.033343    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:52.180651 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.180620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"]
Jul 08 19:40:52.182712 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:40:52.182678    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0dec084_f070_4a16_b762_759e9326afaa.slice/crio-7ee8023c4cbbff49ca5abf5336cba76cd42bc3a3177d74b0423fe586a8ca3e7b WatchSource:0}: Error finding container 7ee8023c4cbbff49ca5abf5336cba76cd42bc3a3177d74b0423fe586a8ca3e7b: Status 404 returned error can't find the container with id 7ee8023c4cbbff49ca5abf5336cba76cd42bc3a3177d74b0423fe586a8ca3e7b
Jul 08 19:40:52.185169 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.185130    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:40:52.185241 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.185227    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:40:52.185274 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.185256    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:40:52.558277 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.558246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" event={"ID":"b0dec084-f070-4a16-b762-759e9326afaa","Type":"ContainerStarted","Data":"037d2b6da0d714583a22f29af4b581717fd7d55693018fbf393d48a9243dd402"}
Jul 08 19:40:52.558461 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.558299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" event={"ID":"b0dec084-f070-4a16-b762-759e9326afaa","Type":"ContainerStarted","Data":"7ee8023c4cbbff49ca5abf5336cba76cd42bc3a3177d74b0423fe586a8ca3e7b"}
Jul 08 19:40:52.581264 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:52.581157    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podStartSLOduration=1.581141302 podStartE2EDuration="1.581141302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:40:52.579401194 +0000 UTC m=+2126.243963759" watchObservedRunningTime="2026-07-08 19:40:52.581141302 +0000 UTC m=+2126.245703851"
Jul 08 19:40:53.033728 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:53.033688    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:53.038854 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:53.038825    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:53.564155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:53.564119    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:53.565036 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:53.565016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:40:54.543778 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.543750    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:40:54.738906 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.738873    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"]
Jul 08 19:40:54.739448 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.739428    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb8260c-5b0b-4a48-aea4-501901b7380e" containerName="istio-proxy"
Jul 08 19:40:54.742929 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.742904    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.745629 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.745604    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e62be2-serving-cert\""
Jul 08 19:40:54.753709 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.753684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"]
Jul 08 19:40:54.809702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.809596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbt87\" (UniqueName: \"kubernetes.io/projected/ef723760-571b-4425-aba5-b06e6a4345c7-kube-api-access-kbt87\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.809702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.809677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef723760-571b-4425-aba5-b06e6a4345c7-openshift-service-ca\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.809702 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.809698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.910477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.910434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbt87\" (UniqueName: \"kubernetes.io/projected/ef723760-571b-4425-aba5-b06e6a4345c7-kube-api-access-kbt87\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.910683 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.910538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef723760-571b-4425-aba5-b06e6a4345c7-openshift-service-ca\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.910683 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.910568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.910849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:54.910728    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-serving-cert: secret "maas-api-e2e-api-b-e62be2-serving-cert" not found
Jul 08 19:40:54.910849 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:40:54.910830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls podName:ef723760-571b-4425-aba5-b06e6a4345c7 nodeName:}" failed. No retries permitted until 2026-07-08 19:40:55.410806175 +0000 UTC m=+2129.075368707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls") pod "maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" (UID: "ef723760-571b-4425-aba5-b06e6a4345c7") : secret "maas-api-e2e-api-b-e62be2-serving-cert" not found
Jul 08 19:40:54.911189 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.911170    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef723760-571b-4425-aba5-b06e6a4345c7-openshift-service-ca\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:54.923869 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:54.923826    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbt87\" (UniqueName: \"kubernetes.io/projected/ef723760-571b-4425-aba5-b06e6a4345c7-kube-api-access-kbt87\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:55.415598 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:55.415552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:55.418546 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:55.418519    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls\") pod \"maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:55.657201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:55.657151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:40:55.857666 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:55.857621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"]
Jul 08 19:40:55.859000 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:40:55.858949    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef723760_571b_4425_aba5_b06e6a4345c7.slice/crio-06bee0d8244f27f8d78a0224a354d021dfa9dc320c90ecdb96cfee84c1df92d9 WatchSource:0}: Error finding container 06bee0d8244f27f8d78a0224a354d021dfa9dc320c90ecdb96cfee84c1df92d9: Status 404 returned error can't find the container with id 06bee0d8244f27f8d78a0224a354d021dfa9dc320c90ecdb96cfee84c1df92d9
Jul 08 19:40:56.583695 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:56.583588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" event={"ID":"ef723760-571b-4425-aba5-b06e6a4345c7","Type":"ContainerStarted","Data":"f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb"}
Jul 08 19:40:56.583695 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:56.583658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" event={"ID":"ef723760-571b-4425-aba5-b06e6a4345c7","Type":"ContainerStarted","Data":"06bee0d8244f27f8d78a0224a354d021dfa9dc320c90ecdb96cfee84c1df92d9"}
Jul 08 19:40:56.584596 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:40:56.584567    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:41:02.437671 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.437607    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": read tcp 10.134.0.2:53068->10.134.0.60:15021: read: connection reset by peer" start-of-body=
Jul 08 19:41:02.438045 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.437727    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": read tcp 10.134.0.2:53068->10.134.0.60:15021: read: connection reset by peer"
Jul 08 19:41:02.627384 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.627347    2563 generic.go:356] "Generic (PLEG): container finished" podID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerID="7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b" exitCode=0
Jul 08 19:41:02.627575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.627425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" event={"ID":"ed41d798-c5e3-477c-bcee-dc541c214b0b","Type":"ContainerDied","Data":"7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"}
Jul 08 19:41:02.627909 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.627896    2563 scope.go:117] "RemoveContainer" containerID="7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"
Jul 08 19:41:02.629046 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.629017    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:02.629163 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.629140    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:02.629203 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.629187    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:02.649564 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:02.649511    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" podStartSLOduration=8.350381155 podStartE2EDuration="8.649487311s" totalImagesPullingTime="299.106156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:40:56.604735169 +0000 UTC m=+2130.269297730" watchObservedRunningTime="2026-07-08 19:41:02.649487311 +0000 UTC m=+2136.314049910"
Jul 08 19:41:03.612421 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:03.612379    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:41:03.641472 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:03.641427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" event={"ID":"ed41d798-c5e3-477c-bcee-dc541c214b0b","Type":"ContainerStarted","Data":"f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322"}
Jul 08 19:41:03.666845 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:03.666804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:41:03.667039 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:03.667017    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 19:41:03.667111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:03.667073    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 19:41:04.667994 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:04.667956    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 19:41:04.668446 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:04.668015    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 19:41:05.667580 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:05.667530    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 19:41:05.667767 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:05.667617    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 19:41:06.667246 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:06.667203    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 19:41:06.667667 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:06.667277    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 19:41:07.667667 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:07.667634    2563 patch_prober.go:28] interesting pod/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 19:41:07.668180 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:07.668152    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 19:41:08.196206 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.196158    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"]
Jul 08 19:41:08.196573 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.196502    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m" podUID="9b9ae576-6037-4f10-99eb-ac53d222ef59" containerName="maas-api" containerID="cri-o://6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377" gracePeriod=30
Jul 08 19:41:08.468911 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.468881    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:41:08.663122 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.663051    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9ae576-6037-4f10-99eb-ac53d222ef59-openshift-service-ca\") pod \"9b9ae576-6037-4f10-99eb-ac53d222ef59\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") "
Jul 08 19:41:08.663296 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.663284    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b9ae576-6037-4f10-99eb-ac53d222ef59-maas-api-tls\") pod \"9b9ae576-6037-4f10-99eb-ac53d222ef59\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") "
Jul 08 19:41:08.663341 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.663332    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4tfd\" (UniqueName: \"kubernetes.io/projected/9b9ae576-6037-4f10-99eb-ac53d222ef59-kube-api-access-j4tfd\") pod \"9b9ae576-6037-4f10-99eb-ac53d222ef59\" (UID: \"9b9ae576-6037-4f10-99eb-ac53d222ef59\") "
Jul 08 19:41:08.663474 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.663412    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b9ae576-6037-4f10-99eb-ac53d222ef59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b9ae576-6037-4f10-99eb-ac53d222ef59" (UID: "9b9ae576-6037-4f10-99eb-ac53d222ef59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:41:08.663771 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.663747    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9ae576-6037-4f10-99eb-ac53d222ef59-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:08.665680 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.665648    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b9ae576-6037-4f10-99eb-ac53d222ef59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b9ae576-6037-4f10-99eb-ac53d222ef59" (UID: "9b9ae576-6037-4f10-99eb-ac53d222ef59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:41:08.665944 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.665918    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9ae576-6037-4f10-99eb-ac53d222ef59-kube-api-access-j4tfd" (OuterVolumeSpecName: "kube-api-access-j4tfd") pod "9b9ae576-6037-4f10-99eb-ac53d222ef59" (UID: "9b9ae576-6037-4f10-99eb-ac53d222ef59"). InnerVolumeSpecName "kube-api-access-j4tfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:41:08.669795 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.669753    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b9ae576-6037-4f10-99eb-ac53d222ef59" containerID="6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377" exitCode=0
Jul 08 19:41:08.670228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.669809    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"
Jul 08 19:41:08.670228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.669804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m" event={"ID":"9b9ae576-6037-4f10-99eb-ac53d222ef59","Type":"ContainerDied","Data":"6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377"}
Jul 08 19:41:08.670228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.669876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m" event={"ID":"9b9ae576-6037-4f10-99eb-ac53d222ef59","Type":"ContainerDied","Data":"21ecf3bee6e8be80717aca300f4ee72e6acd5392fe90566f417a84a0994976b3"}
Jul 08 19:41:08.670228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.669905    2563 scope.go:117] "RemoveContainer" containerID="6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377"
Jul 08 19:41:08.672447 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.672423    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:41:08.672799 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.672768    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:41:08.675132 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.675076    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:41:08.689330 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.689307    2563 scope.go:117] "RemoveContainer" containerID="6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377"
Jul 08 19:41:08.689692 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:41:08.689665    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377\": container with ID starting with 6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377 not found: ID does not exist" containerID="6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377"
Jul 08 19:41:08.689744 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.689706    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377"} err="failed to get container status \"6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377\": rpc error: code = NotFound desc = could not find container \"6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377\": container with ID starting with 6a7d063b655c6d0f6658680f66263546079ddccb9140699f1fb01c92ffa0f377 not found: ID does not exist"
Jul 08 19:41:08.732910 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.732812    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"]
Jul 08 19:41:08.738225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.736739    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8627fc-64bf954c-nvx9m"]
Jul 08 19:41:08.764672 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.764640    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b9ae576-6037-4f10-99eb-ac53d222ef59-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:08.764672 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.764667    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4tfd\" (UniqueName: \"kubernetes.io/projected/9b9ae576-6037-4f10-99eb-ac53d222ef59-kube-api-access-j4tfd\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:08.931739 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:08.931702    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b9ae576-6037-4f10-99eb-ac53d222ef59" path="/var/lib/kubelet/pods/9b9ae576-6037-4f10-99eb-ac53d222ef59/volumes"
Jul 08 19:41:09.605331 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:09.605285    2563 patch_prober.go:28] interesting pod/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 19:41:09.605502 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:09.605354    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 19:41:11.689991 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:11.689888    2563 generic.go:356] "Generic (PLEG): container finished" podID="b0dec084-f070-4a16-b762-759e9326afaa" containerID="037d2b6da0d714583a22f29af4b581717fd7d55693018fbf393d48a9243dd402" exitCode=0
Jul 08 19:41:11.689991 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:11.689973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" event={"ID":"b0dec084-f070-4a16-b762-759e9326afaa","Type":"ContainerDied","Data":"037d2b6da0d714583a22f29af4b581717fd7d55693018fbf393d48a9243dd402"}
Jul 08 19:41:11.690603 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:11.690544    2563 scope.go:117] "RemoveContainer" containerID="037d2b6da0d714583a22f29af4b581717fd7d55693018fbf393d48a9243dd402"
Jul 08 19:41:11.691727 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:11.691698    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:11.691833 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:11.691799    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:11.691907 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:11.691845    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:12.702849 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:12.702796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" event={"ID":"b0dec084-f070-4a16-b762-759e9326afaa","Type":"ContainerStarted","Data":"3bec339a5d856fb8a7a2e6b45cb4a4aa9ef2f5ff4bfc1579ab71f078c3c2d6af"}
Jul 08 19:41:13.033640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:13.033545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:41:13.033871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:13.033848    2563 patch_prober.go:28] interesting pod/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 19:41:13.033940 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:13.033918    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 19:41:13.715301 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:13.715251    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"]
Jul 08 19:41:13.715863 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:13.715821    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy" containerID="cri-o://f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322" gracePeriod=30
Jul 08 19:41:14.034852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:14.034741    2563 patch_prober.go:28] interesting pod/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 19:41:14.034852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:14.034821    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 19:41:14.944029 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:14.943990    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"]
Jul 08 19:41:14.944525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:14.944286    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" podUID="ef723760-571b-4425-aba5-b06e6a4345c7" containerName="maas-api" containerID="cri-o://f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb" gracePeriod=30
Jul 08 19:41:15.034425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.034009    2563 patch_prober.go:28] interesting pod/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 19:41:15.034425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.034074    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 19:41:15.236028 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.235785    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:41:15.332864 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.332815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls\") pod \"ef723760-571b-4425-aba5-b06e6a4345c7\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") "
Jul 08 19:41:15.333116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.332900    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef723760-571b-4425-aba5-b06e6a4345c7-openshift-service-ca\") pod \"ef723760-571b-4425-aba5-b06e6a4345c7\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") "
Jul 08 19:41:15.333116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.333006    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbt87\" (UniqueName: \"kubernetes.io/projected/ef723760-571b-4425-aba5-b06e6a4345c7-kube-api-access-kbt87\") pod \"ef723760-571b-4425-aba5-b06e6a4345c7\" (UID: \"ef723760-571b-4425-aba5-b06e6a4345c7\") "
Jul 08 19:41:15.333334 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.333288    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef723760-571b-4425-aba5-b06e6a4345c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef723760-571b-4425-aba5-b06e6a4345c7" (UID: "ef723760-571b-4425-aba5-b06e6a4345c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:41:15.335264 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.335238    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef723760-571b-4425-aba5-b06e6a4345c7" (UID: "ef723760-571b-4425-aba5-b06e6a4345c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:41:15.335358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.335313    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef723760-571b-4425-aba5-b06e6a4345c7-kube-api-access-kbt87" (OuterVolumeSpecName: "kube-api-access-kbt87") pod "ef723760-571b-4425-aba5-b06e6a4345c7" (UID: "ef723760-571b-4425-aba5-b06e6a4345c7"). InnerVolumeSpecName "kube-api-access-kbt87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:41:15.434207 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.434171    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef723760-571b-4425-aba5-b06e6a4345c7-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:15.434207 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.434201    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbt87\" (UniqueName: \"kubernetes.io/projected/ef723760-571b-4425-aba5-b06e6a4345c7-kube-api-access-kbt87\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:15.434207 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.434211    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef723760-571b-4425-aba5-b06e6a4345c7-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:15.723129 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.723015    2563 generic.go:356] "Generic (PLEG): container finished" podID="ef723760-571b-4425-aba5-b06e6a4345c7" containerID="f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb" exitCode=0
Jul 08 19:41:15.723129 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.723076    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"
Jul 08 19:41:15.723370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.723157    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" event={"ID":"ef723760-571b-4425-aba5-b06e6a4345c7","Type":"ContainerDied","Data":"f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb"}
Jul 08 19:41:15.723370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.723225    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc" event={"ID":"ef723760-571b-4425-aba5-b06e6a4345c7","Type":"ContainerDied","Data":"06bee0d8244f27f8d78a0224a354d021dfa9dc320c90ecdb96cfee84c1df92d9"}
Jul 08 19:41:15.723370 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.723254    2563 scope.go:117] "RemoveContainer" containerID="f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb"
Jul 08 19:41:15.733113 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.733067    2563 scope.go:117] "RemoveContainer" containerID="f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb"
Jul 08 19:41:15.733425 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:41:15.733404    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb\": container with ID starting with f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb not found: ID does not exist" containerID="f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb"
Jul 08 19:41:15.733515 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.733441    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb"} err="failed to get container status \"f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb\": rpc error: code = NotFound desc = could not find container \"f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb\": container with ID starting with f6d188840d402452ded31224ca4b27a27b22f3aaf857e6e82ab37667c45fb5eb not found: ID does not exist"
Jul 08 19:41:15.745475 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.745442    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"]
Jul 08 19:41:15.754202 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:15.754173    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e62be2-657656d5b4-kqqtc"]
Jul 08 19:41:16.033960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:16.033857    2563 patch_prober.go:28] interesting pod/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 19:41:16.033960 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:16.033923    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 19:41:16.931986 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:16.931903    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef723760-571b-4425-aba5-b06e6a4345c7" path="/var/lib/kubelet/pods/ef723760-571b-4425-aba5-b06e6a4345c7/volumes"
Jul 08 19:41:17.038097 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:17.038047    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:41:17.038496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:17.038343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:41:17.039044 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:17.039025    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:41:18.995037 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:18.995007    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:41:19.070556 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-token\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.070774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-certs\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.070774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-credential-socket\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.070774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-envoy\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.070774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070672    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-data\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.070774 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed41d798-c5e3-477c-bcee-dc541c214b0b-istiod-ca-cert\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.071059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070791    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-socket\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.071059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070843    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-podinfo\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.071059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.070891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsrtf\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-kube-api-access-hsrtf\") pod \"ed41d798-c5e3-477c-bcee-dc541c214b0b\" (UID: \"ed41d798-c5e3-477c-bcee-dc541c214b0b\") "
Jul 08 19:41:19.071059 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071008    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:19.071312 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071185    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:19.071364 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071319    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.071364 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071342    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.071496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071473    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:19.071718 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071693    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed41d798-c5e3-477c-bcee-dc541c214b0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:41:19.071806 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.071715    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:19.073031 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.073001    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:41:19.073205 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.073185    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-kube-api-access-hsrtf" (OuterVolumeSpecName: "kube-api-access-hsrtf") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "kube-api-access-hsrtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:41:19.073471 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.073453    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:19.073513 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.073479    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed41d798-c5e3-477c-bcee-dc541c214b0b" (UID: "ed41d798-c5e3-477c-bcee-dc541c214b0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:41:19.172117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172036    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.172117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172073    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed41d798-c5e3-477c-bcee-dc541c214b0b-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.172117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172116    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.172117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172124    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsrtf\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-kube-api-access-hsrtf\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.172117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172134    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.172427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172142    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.172427 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.172149    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed41d798-c5e3-477c-bcee-dc541c214b0b-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:19.747360 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.747321    2563 generic.go:356] "Generic (PLEG): container finished" podID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerID="f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322" exitCode=0
Jul 08 19:41:19.747575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.747379    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"
Jul 08 19:41:19.747575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.747407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" event={"ID":"ed41d798-c5e3-477c-bcee-dc541c214b0b","Type":"ContainerDied","Data":"f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322"}
Jul 08 19:41:19.747575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.747473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2" event={"ID":"ed41d798-c5e3-477c-bcee-dc541c214b0b","Type":"ContainerDied","Data":"9608f760e9239cfcfd32d61e86bfa71440bb5dcf0fb670d6eb7605df13ff10aa"}
Jul 08 19:41:19.747575 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.747503    2563 scope.go:117] "RemoveContainer" containerID="f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322"
Jul 08 19:41:19.757425 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.757406    2563 scope.go:117] "RemoveContainer" containerID="7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"
Jul 08 19:41:19.766273 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.766251    2563 scope.go:117] "RemoveContainer" containerID="f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322"
Jul 08 19:41:19.766587 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:41:19.766566    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322\": container with ID starting with f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322 not found: ID does not exist" containerID="f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322"
Jul 08 19:41:19.766634 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.766597    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322"} err="failed to get container status \"f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322\": rpc error: code = NotFound desc = could not find container \"f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322\": container with ID starting with f9a01dec2882cf5066b34617ad2b4ce859d846029fee9b5f0cf29d2399cc5322 not found: ID does not exist"
Jul 08 19:41:19.766634 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.766617    2563 scope.go:117] "RemoveContainer" containerID="7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"
Jul 08 19:41:19.766899 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:41:19.766882    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b\": container with ID starting with 7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b not found: ID does not exist" containerID="7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"
Jul 08 19:41:19.766987 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.766906    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b"} err="failed to get container status \"7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b\": rpc error: code = NotFound desc = could not find container \"7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b\": container with ID starting with 7f232bc2f266f026a9919d09856f8017ff43ba855cc1a736a5e79b33dd171b5b not found: ID does not exist"
Jul 08 19:41:19.770279 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.770253    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"]
Jul 08 19:41:19.774626 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:19.774601    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8627fc-openshift-default-78f7f8d7ff-7hfk2"]
Jul 08 19:41:20.574588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:20.574555    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"]
Jul 08 19:41:20.575072 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:20.574792    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy" containerID="cri-o://3bec339a5d856fb8a7a2e6b45cb4a4aa9ef2f5ff4bfc1579ab71f078c3c2d6af" gracePeriod=30
Jul 08 19:41:20.931035 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:20.931002    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" path="/var/lib/kubelet/pods/ed41d798-c5e3-477c-bcee-dc541c214b0b/volumes"
Jul 08 19:41:21.438155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438117    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"]
Jul 08 19:41:21.438633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438615    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef723760-571b-4425-aba5-b06e6a4345c7" containerName="maas-api"
Jul 08 19:41:21.438633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438634    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef723760-571b-4425-aba5-b06e6a4345c7" containerName="maas-api"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438651    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438660    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438702    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9ae576-6037-4f10-99eb-ac53d222ef59" containerName="maas-api"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438708    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9ae576-6037-4f10-99eb-ac53d222ef59" containerName="maas-api"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438715    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438741    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438803    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef723760-571b-4425-aba5-b06e6a4345c7" containerName="maas-api"
Jul 08 19:41:21.438807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438810    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy"
Jul 08 19:41:21.439229 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.438817    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b9ae576-6037-4f10-99eb-ac53d222ef59" containerName="maas-api"
Jul 08 19:41:21.442094 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.442057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.444851 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.444833    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-82d12c-openshift-default-dockercfg-bds29\""
Jul 08 19:41:21.454563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.454517    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"]
Jul 08 19:41:21.494862 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.494824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-podinfo\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.494883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mjm\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-kube-api-access-f9mjm\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.494994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-token\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495062 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.495044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istiod-ca-cert\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.495128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-envoy\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495222 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.495199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-socket\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.495227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-data\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.495265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-certs\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.495379 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.495328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-credential-socket\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596157 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-token\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istiod-ca-cert\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-envoy\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-socket\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-data\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-certs\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-credential-socket\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-podinfo\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mjm\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-kube-api-access-f9mjm\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596732    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-socket\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596769    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-certs\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.596980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596821    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-credential-socket\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.597253 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.596985    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-data\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.597253 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.597161    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istiod-ca-cert\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.599181 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.599154    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-envoy\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.599394 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.599372    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-podinfo\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.605004 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.604978    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mjm\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-kube-api-access-f9mjm\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.605133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.604981    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-token\") pod \"e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") " pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.754783 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.754657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:21.912579 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.912540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"]
Jul 08 19:41:21.914282 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:41:21.914251    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff1b4df_ece7_4d37_b0a7_f24bfa806b4d.slice/crio-7afbc7d903594f165919c2d9ca2c89bc080070e2e8483f5156bfdf643be8310f WatchSource:0}: Error finding container 7afbc7d903594f165919c2d9ca2c89bc080070e2e8483f5156bfdf643be8310f: Status 404 returned error can't find the container with id 7afbc7d903594f165919c2d9ca2c89bc080070e2e8483f5156bfdf643be8310f
Jul 08 19:41:21.916735 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.916696    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:21.916856 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.916786    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:21.916856 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:21.916834    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:22.773574 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:22.773531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" event={"ID":"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d","Type":"ContainerStarted","Data":"023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"}
Jul 08 19:41:22.774063 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:22.773596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" event={"ID":"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d","Type":"ContainerStarted","Data":"7afbc7d903594f165919c2d9ca2c89bc080070e2e8483f5156bfdf643be8310f"}
Jul 08 19:41:22.795638 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:22.795587    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" podStartSLOduration=1.7955679409999998 podStartE2EDuration="1.795567941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:22.794479652 +0000 UTC m=+2156.459042203" watchObservedRunningTime="2026-07-08 19:41:22.795567941 +0000 UTC m=+2156.460130491"
Jul 08 19:41:23.755375 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:23.755335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:23.760460 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:23.760433    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:23.779555 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:23.779516    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:23.780640 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:23.780617    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:24.791211 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.791138    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"]
Jul 08 19:41:24.792516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.792419    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed41d798-c5e3-477c-bcee-dc541c214b0b" containerName="istio-proxy"
Jul 08 19:41:24.796677 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.795809    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:24.801105 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.799703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6bnq4\""
Jul 08 19:41:24.801105 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.800033    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-82d12c-serving-cert\""
Jul 08 19:41:24.807473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.807450    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"]
Jul 08 19:41:24.930616 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.930586    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/803f43f2-202b-40ee-8ea4-c2598950f453-openshift-service-ca\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:24.930855 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.930652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:24.930919 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:24.930844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk9ft\" (UniqueName: \"kubernetes.io/projected/803f43f2-202b-40ee-8ea4-c2598950f453-kube-api-access-tk9ft\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.031636 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.031600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk9ft\" (UniqueName: \"kubernetes.io/projected/803f43f2-202b-40ee-8ea4-c2598950f453-kube-api-access-tk9ft\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.031852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.031660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/803f43f2-202b-40ee-8ea4-c2598950f453-openshift-service-ca\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.031852 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.031758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.032382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.032362    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/803f43f2-202b-40ee-8ea4-c2598950f453-openshift-service-ca\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.034516 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.034494    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.040414 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.040389    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk9ft\" (UniqueName: \"kubernetes.io/projected/803f43f2-202b-40ee-8ea4-c2598950f453-kube-api-access-tk9ft\") pod \"maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.116255 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.116228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.262308 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.262285    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"]
Jul 08 19:41:25.264289 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:41:25.264261    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803f43f2_202b_40ee_8ea4_c2598950f453.slice/crio-7e1bf6305aa08a5ba8572a82fe093b69092e90a217039b8e91af0235cf43ea1e WatchSource:0}: Error finding container 7e1bf6305aa08a5ba8572a82fe093b69092e90a217039b8e91af0235cf43ea1e: Status 404 returned error can't find the container with id 7e1bf6305aa08a5ba8572a82fe093b69092e90a217039b8e91af0235cf43ea1e
Jul 08 19:41:25.794633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.794600    2563 generic.go:356] "Generic (PLEG): container finished" podID="b0dec084-f070-4a16-b762-759e9326afaa" containerID="3bec339a5d856fb8a7a2e6b45cb4a4aa9ef2f5ff4bfc1579ab71f078c3c2d6af" exitCode=0
Jul 08 19:41:25.795310 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.794667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" event={"ID":"b0dec084-f070-4a16-b762-759e9326afaa","Type":"ContainerDied","Data":"3bec339a5d856fb8a7a2e6b45cb4a4aa9ef2f5ff4bfc1579ab71f078c3c2d6af"}
Jul 08 19:41:25.795310 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.794758    2563 scope.go:117] "RemoveContainer" containerID="037d2b6da0d714583a22f29af4b581717fd7d55693018fbf393d48a9243dd402"
Jul 08 19:41:25.796807 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.796775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" event={"ID":"803f43f2-202b-40ee-8ea4-c2598950f453","Type":"ContainerStarted","Data":"35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19"}
Jul 08 19:41:25.796945 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.796831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" event={"ID":"803f43f2-202b-40ee-8ea4-c2598950f453","Type":"ContainerStarted","Data":"7e1bf6305aa08a5ba8572a82fe093b69092e90a217039b8e91af0235cf43ea1e"}
Jul 08 19:41:25.797033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.797014    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:25.814633 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.814580    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" podStartSLOduration=1.354349323 podStartE2EDuration="1.814558292s" totalImagesPullingTime="460.208969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:25.813750899 +0000 UTC m=+2159.478313449" watchObservedRunningTime="2026-07-08 19:41:25.814558292 +0000 UTC m=+2159.479120846"
Jul 08 19:41:25.833097 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.833057    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:41:25.946953 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.946891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-envoy\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.946964    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0dec084-f070-4a16-b762-759e9326afaa-istio-podinfo\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947013    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-socket\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-certs\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947160 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947115    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nj9k\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-kube-api-access-4nj9k\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947266    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-data\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947301    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-istio-token\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947327    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:25.947473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947339    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0dec084-f070-4a16-b762-759e9326afaa-istiod-ca-cert\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947421    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-credential-socket\") pod \"b0dec084-f070-4a16-b762-759e9326afaa\" (UID: \"b0dec084-f070-4a16-b762-759e9326afaa\") "
Jul 08 19:41:25.947705 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947632    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0dec084-f070-4a16-b762-759e9326afaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:41:25.947870 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947837    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0dec084-f070-4a16-b762-759e9326afaa-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:25.947964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947875    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:25.947964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947882    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:25.948098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.947988    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:25.948098 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.948017    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:25.949536 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.949512    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0dec084-f070-4a16-b762-759e9326afaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:41:25.949848 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.949817    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:41:25.949848 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.949840    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:41:25.949986 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:25.949969    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-kube-api-access-4nj9k" (OuterVolumeSpecName: "kube-api-access-4nj9k") pod "b0dec084-f070-4a16-b762-759e9326afaa" (UID: "b0dec084-f070-4a16-b762-759e9326afaa"). InnerVolumeSpecName "kube-api-access-4nj9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:41:26.049012 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.048978    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nj9k\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-kube-api-access-4nj9k\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.049012 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.049011    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.049251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.049021    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0dec084-f070-4a16-b762-759e9326afaa-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.049251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.049028    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.049251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.049037    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.049251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.049044    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0dec084-f070-4a16-b762-759e9326afaa-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.049251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.049051    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0dec084-f070-4a16-b762-759e9326afaa-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:41:26.803284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.803250    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"
Jul 08 19:41:26.803284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.803273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79" event={"ID":"b0dec084-f070-4a16-b762-759e9326afaa","Type":"ContainerDied","Data":"7ee8023c4cbbff49ca5abf5336cba76cd42bc3a3177d74b0423fe586a8ca3e7b"}
Jul 08 19:41:26.803813 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.803340    2563 scope.go:117] "RemoveContainer" containerID="3bec339a5d856fb8a7a2e6b45cb4a4aa9ef2f5ff4bfc1579ab71f078c3c2d6af"
Jul 08 19:41:26.827527 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.827497    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"]
Jul 08 19:41:26.831329 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.831300    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e62be2-openshift-default-7f5d77d7d5-2sb79"]
Jul 08 19:41:26.930385 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:26.930353    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0dec084-f070-4a16-b762-759e9326afaa" path="/var/lib/kubelet/pods/b0dec084-f070-4a16-b762-759e9326afaa/volumes"
Jul 08 19:41:29.873827 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.873795    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"]
Jul 08 19:41:29.874379 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.874275    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy"
Jul 08 19:41:29.874379 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.874292    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy"
Jul 08 19:41:29.874379 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.874370    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy"
Jul 08 19:41:29.876935 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.876915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.880884 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.880855    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-235b05-openshift-default-dockercfg-pp28c\""
Jul 08 19:41:29.888136 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.888106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"]
Jul 08 19:41:29.988557 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-certs\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988739 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-podinfo\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988739 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmfv2\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-kube-api-access-wmfv2\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988861 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-credential-socket\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988861 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-token\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988861 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-socket\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istiod-ca-cert\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.988982 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-envoy\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:29.989051 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:29.988987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-data\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.090541 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-credential-socket\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.090755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-token\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.090755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-socket\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.090755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istiod-ca-cert\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.090755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-envoy\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.090755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-data\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-certs\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-podinfo\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.090813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmfv2\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-kube-api-access-wmfv2\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091386 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.091299    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-socket\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091386 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.091331    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-credential-socket\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091531 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.091422    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-certs\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091708 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.091688    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-data\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.091918 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.091894    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istiod-ca-cert\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.093220 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.093196    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-envoy\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.093410 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.093391    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-podinfo\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.099155 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.099125    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-token\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.099359 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.099341    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmfv2\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-kube-api-access-wmfv2\") pod \"e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") " pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.190408 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.190312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:30.348161 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.348124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"]
Jul 08 19:41:30.349295 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:41:30.349268    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aaa24f0_9d54_4009_9e42_ca9071bea0cb.slice/crio-2d80927244ba6ad0efe3893228b29bc261b0c465eaf24b2353a0c78797abe087 WatchSource:0}: Error finding container 2d80927244ba6ad0efe3893228b29bc261b0c465eaf24b2353a0c78797abe087: Status 404 returned error can't find the container with id 2d80927244ba6ad0efe3893228b29bc261b0c465eaf24b2353a0c78797abe087
Jul 08 19:41:30.351866 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.351818    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:30.351942 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.351898    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:30.351942 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.351923    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:30.825429 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.825392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" event={"ID":"1aaa24f0-9d54-4009-9e42-ca9071bea0cb","Type":"ContainerStarted","Data":"ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"}
Jul 08 19:41:30.825646 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.825473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" event={"ID":"1aaa24f0-9d54-4009-9e42-ca9071bea0cb","Type":"ContainerStarted","Data":"2d80927244ba6ad0efe3893228b29bc261b0c465eaf24b2353a0c78797abe087"}
Jul 08 19:41:30.871134 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:30.871060    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podStartSLOduration=1.8710394940000001 podStartE2EDuration="1.871039494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:30.866464428 +0000 UTC m=+2164.531027140" watchObservedRunningTime="2026-07-08 19:41:30.871039494 +0000 UTC m=+2164.535602047"
Jul 08 19:41:31.193382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:31.191488    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:31.205280 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:31.205247    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:31.809741 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:31.809708    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:41:31.831668 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:31.831636    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:31.832611 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:31.832589    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:32.873609 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.873568    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"]
Jul 08 19:41:32.874228 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.874211    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy"
Jul 08 19:41:32.874293 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.874231    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy"
Jul 08 19:41:32.874330 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.874321    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0dec084-f070-4a16-b762-759e9326afaa" containerName="istio-proxy"
Jul 08 19:41:32.905120 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.904115    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"]
Jul 08 19:41:32.905120 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.904246    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:32.910470 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:32.910439    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-235b05-serving-cert\""
Jul 08 19:41:33.024734 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.024699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007d7ba6-ead6-44a2-ab92-591f0bb4720a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.024903 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.024760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhtmw\" (UniqueName: \"kubernetes.io/projected/007d7ba6-ead6-44a2-ab92-591f0bb4720a-kube-api-access-hhtmw\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.024964 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.024920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.126170 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.126028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.126358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.126194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007d7ba6-ead6-44a2-ab92-591f0bb4720a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.126358 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:41:33.126226    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-serving-cert: secret "maas-api-e2e-shared-b-235b05-serving-cert" not found
Jul 08 19:41:33.126358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.126263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhtmw\" (UniqueName: \"kubernetes.io/projected/007d7ba6-ead6-44a2-ab92-591f0bb4720a-kube-api-access-hhtmw\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.126535 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:41:33.126372    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls podName:007d7ba6-ead6-44a2-ab92-591f0bb4720a nodeName:}" failed. No retries permitted until 2026-07-08 19:41:33.626288665 +0000 UTC m=+2167.290851205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls") pod "maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" (UID: "007d7ba6-ead6-44a2-ab92-591f0bb4720a") : secret "maas-api-e2e-shared-b-235b05-serving-cert" not found
Jul 08 19:41:33.127065 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.127041    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007d7ba6-ead6-44a2-ab92-591f0bb4720a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.137315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.137294    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhtmw\" (UniqueName: \"kubernetes.io/projected/007d7ba6-ead6-44a2-ab92-591f0bb4720a-kube-api-access-hhtmw\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.631733 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.631684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.641617 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.641590    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls\") pod \"maas-api-e2e-shared-b-235b05-cb96df448-lgfjh\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:33.824829 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:33.824784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:34.011521 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:34.011492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"]
Jul 08 19:41:34.013428 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:41:34.013397    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007d7ba6_ead6_44a2_ab92_591f0bb4720a.slice/crio-4805e48d96980e32b11156947d189f70c5ac7a0dc11a68f3dfef2ccfe5573421 WatchSource:0}: Error finding container 4805e48d96980e32b11156947d189f70c5ac7a0dc11a68f3dfef2ccfe5573421: Status 404 returned error can't find the container with id 4805e48d96980e32b11156947d189f70c5ac7a0dc11a68f3dfef2ccfe5573421
Jul 08 19:41:34.850358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:34.850268    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" event={"ID":"007d7ba6-ead6-44a2-ab92-591f0bb4720a","Type":"ContainerStarted","Data":"fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b"}
Jul 08 19:41:34.850358 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:34.850320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" event={"ID":"007d7ba6-ead6-44a2-ab92-591f0bb4720a","Type":"ContainerStarted","Data":"4805e48d96980e32b11156947d189f70c5ac7a0dc11a68f3dfef2ccfe5573421"}
Jul 08 19:41:34.850552 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:34.850366    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:34.868893 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:34.868834    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" podStartSLOduration=2.459987833 podStartE2EDuration="2.868817445s" totalImagesPullingTime="408.829612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:34.866923789 +0000 UTC m=+2168.531486341" watchObservedRunningTime="2026-07-08 19:41:34.868817445 +0000 UTC m=+2168.533379995"
Jul 08 19:41:37.872716 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:37.872684    2563 generic.go:356] "Generic (PLEG): container finished" podID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerID="023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b" exitCode=0
Jul 08 19:41:37.873274 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:37.872758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" event={"ID":"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d","Type":"ContainerDied","Data":"023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"}
Jul 08 19:41:37.873352 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:37.873310    2563 scope.go:117] "RemoveContainer" containerID="023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"
Jul 08 19:41:37.874357 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:37.874332    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:37.874435 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:37.874398    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:37.874435 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:37.874428    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:38.779748 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:38.779693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:38.883369 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:38.883321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" event={"ID":"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d","Type":"ContainerStarted","Data":"94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75"}
Jul 08 19:41:39.755750 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:39.755708    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:39.756643 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:39.756617    2563 patch_prober.go:28] interesting pod/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 08 19:41:39.756764 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:39.756667    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 08 19:41:40.755382 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:40.755338    2563 patch_prober.go:28] interesting pod/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 08 19:41:40.755801 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:40.755400    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 08 19:41:40.862637 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:40.862603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:41:41.755199 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:41.755158    2563 patch_prober.go:28] interesting pod/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 08 19:41:41.755411 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:41.755223    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 08 19:41:42.760216 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:42.760183    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:42.760685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:42.760497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:42.761311 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:42.761294    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:41:47.874252 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:47.874210    2563 patch_prober.go:28] interesting pod/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 19:41:47.874792 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:47.874272    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 19:41:48.943470 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:48.943435    2563 generic.go:356] "Generic (PLEG): container finished" podID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerID="ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124" exitCode=0
Jul 08 19:41:48.943965 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:48.943507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" event={"ID":"1aaa24f0-9d54-4009-9e42-ca9071bea0cb","Type":"ContainerDied","Data":"ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"}
Jul 08 19:41:48.943965 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:48.943945    2563 scope.go:117] "RemoveContainer" containerID="ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"
Jul 08 19:41:48.944946 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:48.944915    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:48.945037 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:48.944996    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:48.945037 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:48.945026    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 19:41:49.950712 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:49.950668    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" event={"ID":"1aaa24f0-9d54-4009-9e42-ca9071bea0cb","Type":"ContainerStarted","Data":"ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec"}
Jul 08 19:41:50.191168 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:50.191118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:50.191407 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:50.191305    2563 patch_prober.go:28] interesting pod/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 08 19:41:50.191407 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:50.191358    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 08 19:41:51.190821 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:51.190774    2563 patch_prober.go:28] interesting pod/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 08 19:41:51.191364 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:51.190871    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 08 19:41:52.191685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:52.191635    2563 patch_prober.go:28] interesting pod/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 08 19:41:52.192257 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:52.191715    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 08 19:41:53.191133 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:53.191065    2563 patch_prober.go:28] interesting pod/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 08 19:41:53.191442 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:53.191411    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 08 19:41:54.194181 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:54.194147    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:54.194570 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:54.194422    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:41:54.195318 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:41:54.195303    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:44:42.978684 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:42.978640    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"]
Jul 08 19:44:43.022953 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:43.022903    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-serving-cert: secret "maas-api-e2e-shared-a-82d12c-serving-cert" not found
Jul 08 19:44:43.023256 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:43.023019    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls podName:803f43f2-202b-40ee-8ea4-c2598950f453 nodeName:}" failed. No retries permitted until 2026-07-08 19:44:43.522990857 +0000 UTC m=+2357.187553407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls") pod "maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" (UID: "803f43f2-202b-40ee-8ea4-c2598950f453") : secret "maas-api-e2e-shared-a-82d12c-serving-cert" not found
Jul 08 19:44:43.528665 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:43.528622    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-serving-cert: secret "maas-api-e2e-shared-a-82d12c-serving-cert" not found
Jul 08 19:44:43.528857 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:43.528713    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls podName:803f43f2-202b-40ee-8ea4-c2598950f453 nodeName:}" failed. No retries permitted until 2026-07-08 19:44:44.52869677 +0000 UTC m=+2358.193259299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls") pod "maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" (UID: "803f43f2-202b-40ee-8ea4-c2598950f453") : secret "maas-api-e2e-shared-a-82d12c-serving-cert" not found
Jul 08 19:44:43.848432 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:43.848323    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" podUID="803f43f2-202b-40ee-8ea4-c2598950f453" containerName="maas-api" containerID="cri-o://35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19" gracePeriod=30
Jul 08 19:44:44.116956 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.116929    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:44:44.235183 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.235141    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/803f43f2-202b-40ee-8ea4-c2598950f453-openshift-service-ca\") pod \"803f43f2-202b-40ee-8ea4-c2598950f453\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") "
Jul 08 19:44:44.235361 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.235315    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk9ft\" (UniqueName: \"kubernetes.io/projected/803f43f2-202b-40ee-8ea4-c2598950f453-kube-api-access-tk9ft\") pod \"803f43f2-202b-40ee-8ea4-c2598950f453\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") "
Jul 08 19:44:44.235361 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.235342    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls\") pod \"803f43f2-202b-40ee-8ea4-c2598950f453\" (UID: \"803f43f2-202b-40ee-8ea4-c2598950f453\") "
Jul 08 19:44:44.235551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.235528    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/803f43f2-202b-40ee-8ea4-c2598950f453-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "803f43f2-202b-40ee-8ea4-c2598950f453" (UID: "803f43f2-202b-40ee-8ea4-c2598950f453"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:44:44.237538 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.237502    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803f43f2-202b-40ee-8ea4-c2598950f453-kube-api-access-tk9ft" (OuterVolumeSpecName: "kube-api-access-tk9ft") pod "803f43f2-202b-40ee-8ea4-c2598950f453" (UID: "803f43f2-202b-40ee-8ea4-c2598950f453"). InnerVolumeSpecName "kube-api-access-tk9ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:44:44.237669 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.237590    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "803f43f2-202b-40ee-8ea4-c2598950f453" (UID: "803f43f2-202b-40ee-8ea4-c2598950f453"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:44:44.336367 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.336318    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk9ft\" (UniqueName: \"kubernetes.io/projected/803f43f2-202b-40ee-8ea4-c2598950f453-kube-api-access-tk9ft\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:44.336367 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.336363    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/803f43f2-202b-40ee-8ea4-c2598950f453-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:44.336367 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.336375    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/803f43f2-202b-40ee-8ea4-c2598950f453-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:44.854780 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.854744    2563 generic.go:356] "Generic (PLEG): container finished" podID="803f43f2-202b-40ee-8ea4-c2598950f453" containerID="35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19" exitCode=0
Jul 08 19:44:44.854959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.854799    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"
Jul 08 19:44:44.854959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.854832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" event={"ID":"803f43f2-202b-40ee-8ea4-c2598950f453","Type":"ContainerDied","Data":"35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19"}
Jul 08 19:44:44.854959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.854889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk" event={"ID":"803f43f2-202b-40ee-8ea4-c2598950f453","Type":"ContainerDied","Data":"7e1bf6305aa08a5ba8572a82fe093b69092e90a217039b8e91af0235cf43ea1e"}
Jul 08 19:44:44.854959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.854919    2563 scope.go:117] "RemoveContainer" containerID="35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19"
Jul 08 19:44:44.864999 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.864976    2563 scope.go:117] "RemoveContainer" containerID="35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19"
Jul 08 19:44:44.865365 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:44.865344    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19\": container with ID starting with 35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19 not found: ID does not exist" containerID="35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19"
Jul 08 19:44:44.865443 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.865374    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19"} err="failed to get container status \"35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19\": rpc error: code = NotFound desc = could not find container \"35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19\": container with ID starting with 35247d849ae5f2a36d4f028aa5738232b881172866abb7a42de05e3468ad4d19 not found: ID does not exist"
Jul 08 19:44:44.876278 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.876243    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"]
Jul 08 19:44:44.881660 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.881628    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-82d12c-848d48b69d-x7llk"]
Jul 08 19:44:44.930734 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:44.930681    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803f43f2-202b-40ee-8ea4-c2598950f453" path="/var/lib/kubelet/pods/803f43f2-202b-40ee-8ea4-c2598950f453/volumes"
Jul 08 19:44:48.912178 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:48.912133    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"]
Jul 08 19:44:48.912579 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:48.912396    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy" containerID="cri-o://94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75" gracePeriod=30
Jul 08 19:44:50.023672 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.023632    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"]
Jul 08 19:44:50.024645 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.024612    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" podUID="007d7ba6-ead6-44a2-ab92-591f0bb4720a" containerName="maas-api" containerID="cri-o://fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b" gracePeriod=30
Jul 08 19:44:50.278797 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.278716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:44:50.397761 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.397726    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhtmw\" (UniqueName: \"kubernetes.io/projected/007d7ba6-ead6-44a2-ab92-591f0bb4720a-kube-api-access-hhtmw\") pod \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") "
Jul 08 19:44:50.397949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.397818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007d7ba6-ead6-44a2-ab92-591f0bb4720a-openshift-service-ca\") pod \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") "
Jul 08 19:44:50.397949 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.397839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls\") pod \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\" (UID: \"007d7ba6-ead6-44a2-ab92-591f0bb4720a\") "
Jul 08 19:44:50.398245 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.398219    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/007d7ba6-ead6-44a2-ab92-591f0bb4720a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "007d7ba6-ead6-44a2-ab92-591f0bb4720a" (UID: "007d7ba6-ead6-44a2-ab92-591f0bb4720a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:44:50.400047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.400021    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/007d7ba6-ead6-44a2-ab92-591f0bb4720a-kube-api-access-hhtmw" (OuterVolumeSpecName: "kube-api-access-hhtmw") pod "007d7ba6-ead6-44a2-ab92-591f0bb4720a" (UID: "007d7ba6-ead6-44a2-ab92-591f0bb4720a"). InnerVolumeSpecName "kube-api-access-hhtmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:44:50.400047 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.400027    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "007d7ba6-ead6-44a2-ab92-591f0bb4720a" (UID: "007d7ba6-ead6-44a2-ab92-591f0bb4720a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 19:44:50.498848 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.498812    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhtmw\" (UniqueName: \"kubernetes.io/projected/007d7ba6-ead6-44a2-ab92-591f0bb4720a-kube-api-access-hhtmw\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:50.498848 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.498843    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007d7ba6-ead6-44a2-ab92-591f0bb4720a-openshift-service-ca\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:50.498848 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.498854    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007d7ba6-ead6-44a2-ab92-591f0bb4720a-maas-api-tls\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:50.892974 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.892938    2563 generic.go:356] "Generic (PLEG): container finished" podID="007d7ba6-ead6-44a2-ab92-591f0bb4720a" containerID="fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b" exitCode=0
Jul 08 19:44:50.893215 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.892996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" event={"ID":"007d7ba6-ead6-44a2-ab92-591f0bb4720a","Type":"ContainerDied","Data":"fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b"}
Jul 08 19:44:50.893215 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.892999    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"
Jul 08 19:44:50.893215 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.893050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh" event={"ID":"007d7ba6-ead6-44a2-ab92-591f0bb4720a","Type":"ContainerDied","Data":"4805e48d96980e32b11156947d189f70c5ac7a0dc11a68f3dfef2ccfe5573421"}
Jul 08 19:44:50.893215 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.893107    2563 scope.go:117] "RemoveContainer" containerID="fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b"
Jul 08 19:44:50.904049 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.904022    2563 scope.go:117] "RemoveContainer" containerID="fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b"
Jul 08 19:44:50.904489 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:50.904459    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b\": container with ID starting with fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b not found: ID does not exist" containerID="fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b"
Jul 08 19:44:50.904591 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.904501    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b"} err="failed to get container status \"fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b\": rpc error: code = NotFound desc = could not find container \"fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b\": container with ID starting with fbd46029f9532dc0ab7eb079a6eb71a603519e4018937eb9007d1d9fabe94a2b not found: ID does not exist"
Jul 08 19:44:50.920915 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.920877    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"]
Jul 08 19:44:50.922816 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.922788    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-235b05-cb96df448-lgfjh"]
Jul 08 19:44:50.932100 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:50.932052    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="007d7ba6-ead6-44a2-ab92-591f0bb4720a" path="/var/lib/kubelet/pods/007d7ba6-ead6-44a2-ab92-591f0bb4720a/volumes"
Jul 08 19:44:54.168868 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.168839    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:44:54.238662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238563    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istiod-ca-cert\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238608    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-credential-socket\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238662 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-socket\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238673    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-data\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-envoy\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-certs\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9mjm\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-kube-api-access-f9mjm\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.238972 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238947    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-token\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.239225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.238980    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-podinfo\") pod \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\" (UID: \"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d\") "
Jul 08 19:44:54.239225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239006    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:44:54.239225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239021    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:44:54.239225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239139    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:44:54.239225 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239061    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:44:54.239496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239434    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.239496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239453    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.239496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239466    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.239496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239480    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.239942 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.239885    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:44:54.241221 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.241187    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-kube-api-access-f9mjm" (OuterVolumeSpecName: "kube-api-access-f9mjm") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "kube-api-access-f9mjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:44:54.241375 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.241354    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:44:54.241592 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.241576    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:44:54.241653 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.241631    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" (UID: "7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:44:54.340116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.340050    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9mjm\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-kube-api-access-f9mjm\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.340116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.340112    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.340116 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.340127    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.340366 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.340138    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.340366 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.340150    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:44:54.924599 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.924562    2563 generic.go:356] "Generic (PLEG): container finished" podID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerID="94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75" exitCode=0
Jul 08 19:44:54.924779 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.924667    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"
Jul 08 19:44:54.930908 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.930878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" event={"ID":"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d","Type":"ContainerDied","Data":"94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75"}
Jul 08 19:44:54.931111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.930923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn" event={"ID":"7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d","Type":"ContainerDied","Data":"7afbc7d903594f165919c2d9ca2c89bc080070e2e8483f5156bfdf643be8310f"}
Jul 08 19:44:54.931111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.930945    2563 scope.go:117] "RemoveContainer" containerID="94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75"
Jul 08 19:44:54.940449 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.940428    2563 scope.go:117] "RemoveContainer" containerID="023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"
Jul 08 19:44:54.948410 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.948378    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"]
Jul 08 19:44:54.949826 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.949295    2563 scope.go:117] "RemoveContainer" containerID="94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75"
Jul 08 19:44:54.949826 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:54.949792    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75\": container with ID starting with 94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75 not found: ID does not exist" containerID="94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75"
Jul 08 19:44:54.950033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.949829    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75"} err="failed to get container status \"94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75\": rpc error: code = NotFound desc = could not find container \"94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75\": container with ID starting with 94d7bd1a1aa23cea4b75d059f0544a8636b48faf62d5d8cf0fdb5ef4d952ad75 not found: ID does not exist"
Jul 08 19:44:54.950033 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.949857    2563 scope.go:117] "RemoveContainer" containerID="023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"
Jul 08 19:44:54.950534 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:44:54.950512    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b\": container with ID starting with 023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b not found: ID does not exist" containerID="023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"
Jul 08 19:44:54.950612 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.950543    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b"} err="failed to get container status \"023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b\": rpc error: code = NotFound desc = could not find container \"023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b\": container with ID starting with 023f8e07af3477712a3b13f04adbb7dbd92351454361453152d7757bc6b6771b not found: ID does not exist"
Jul 08 19:44:54.952117 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:54.952097    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-82d12c-openshift-default-644b4c8747-prsnn"]
Jul 08 19:44:55.890829 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:55.890754    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"]
Jul 08 19:44:55.891329 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:55.891136    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy" containerID="cri-o://ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec" gracePeriod=30
Jul 08 19:44:56.930469 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:44:56.930431    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" path="/var/lib/kubelet/pods/7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d/volumes"
Jul 08 19:45:00.145052 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145013    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9"]
Jul 08 19:45:00.145477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145460    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy"
Jul 08 19:45:00.145477 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145472    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy"
Jul 08 19:45:00.145563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145498    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="007d7ba6-ead6-44a2-ab92-591f0bb4720a" containerName="maas-api"
Jul 08 19:45:00.145563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145504    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="007d7ba6-ead6-44a2-ab92-591f0bb4720a" containerName="maas-api"
Jul 08 19:45:00.145563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145514    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803f43f2-202b-40ee-8ea4-c2598950f453" containerName="maas-api"
Jul 08 19:45:00.145563 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145520    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="803f43f2-202b-40ee-8ea4-c2598950f453" containerName="maas-api"
Jul 08 19:45:00.145719 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145582    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="803f43f2-202b-40ee-8ea4-c2598950f453" containerName="maas-api"
Jul 08 19:45:00.145719 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145590    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy"
Jul 08 19:45:00.145719 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145596    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy"
Jul 08 19:45:00.145719 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.145602    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="007d7ba6-ead6-44a2-ab92-591f0bb4720a" containerName="maas-api"
Jul 08 19:45:00.148486 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.148469    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9"
Jul 08 19:45:00.156428 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.156402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9"]
Jul 08 19:45:00.159063 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.159040    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9"
Jul 08 19:45:00.303386 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.303357    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9"]
Jul 08 19:45:00.305154 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:45:00.305078    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c14e86_ec25_42e0_99e0_234b5ca41086.slice/crio-161ba8eb6c5af1abe9612935b582878a809b4a102bf79932419306b5bbe917a9 WatchSource:0}: Error finding container 161ba8eb6c5af1abe9612935b582878a809b4a102bf79932419306b5bbe917a9: Status 404 returned error can't find the container with id 161ba8eb6c5af1abe9612935b582878a809b4a102bf79932419306b5bbe917a9
Jul 08 19:45:00.957685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.957648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9" event={"ID":"f6c14e86-ec25-42e0-99e0-234b5ca41086","Type":"ContainerStarted","Data":"42ec8112276d42c2b41ec57d4b8581118993a5e64be11bfb103789331e503cf3"}
Jul 08 19:45:00.957864 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.957717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9" event={"ID":"f6c14e86-ec25-42e0-99e0-234b5ca41086","Type":"ContainerStarted","Data":"161ba8eb6c5af1abe9612935b582878a809b4a102bf79932419306b5bbe917a9"}
Jul 08 19:45:00.974423 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:00.974373    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-kkbj9" podStartSLOduration=0.974356191 podStartE2EDuration="974.356191ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:00.972316978 +0000 UTC m=+2374.636879530" watchObservedRunningTime="2026-07-08 19:45:00.974356191 +0000 UTC m=+2374.638918741"
Jul 08 19:45:01.138003 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.137977    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:45:01.288198 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.288164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2frh5_fea46e71-eb0d-4ad6-96a9-94be06d0d4dd/manager/0.log"
Jul 08 19:45:01.312621 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-podinfo\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.312758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312637    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmfv2\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-kube-api-access-wmfv2\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.312758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-certs\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.312758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-credential-socket\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.312758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-socket\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.312758 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istiod-ca-cert\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.313024 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-envoy\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.313024 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-token\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.313024 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.312886    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-data\") pod \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\" (UID: \"1aaa24f0-9d54-4009-9e42-ca9071bea0cb\") "
Jul 08 19:45:01.313209 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313076    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:45:01.313209 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313105    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:45:01.313315 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313226    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 19:45:01.313351 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313313    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:45:01.313694 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313668    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-certs\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.313819 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313698    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-credential-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.313819 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313716    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-workload-socket\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.313819 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313731    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istiod-ca-cert\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.313956 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.313864    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:45:01.315169 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.315143    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-kube-api-access-wmfv2" (OuterVolumeSpecName: "kube-api-access-wmfv2") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "kube-api-access-wmfv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:45:01.315292 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.315253    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 19:45:01.315551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.315534    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 19:45:01.315551 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.315542    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1aaa24f0-9d54-4009-9e42-ca9071bea0cb" (UID: "1aaa24f0-9d54-4009-9e42-ca9071bea0cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 19:45:01.415060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.415025    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-podinfo\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.415060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.415055    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmfv2\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-kube-api-access-wmfv2\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.415060 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.415065    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-envoy\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.415311 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.415076    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-token\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.415311 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.415110    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaa24f0-9d54-4009-9e42-ca9071bea0cb-istio-data\") on node \"ip-10-0-130-212.ec2.internal\" DevicePath \"\""
Jul 08 19:45:01.531858 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.531752    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-598545dc6b-ncr96_9f4ae60b-2c83-40c5-a2c9-4fb477d05a00/manager/0.log"
Jul 08 19:45:01.964872 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.964833    2563 generic.go:356] "Generic (PLEG): container finished" podID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerID="ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec" exitCode=0
Jul 08 19:45:01.965107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.964896    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"
Jul 08 19:45:01.965107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.964930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" event={"ID":"1aaa24f0-9d54-4009-9e42-ca9071bea0cb","Type":"ContainerDied","Data":"ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec"}
Jul 08 19:45:01.965107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.964990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj" event={"ID":"1aaa24f0-9d54-4009-9e42-ca9071bea0cb","Type":"ContainerDied","Data":"2d80927244ba6ad0efe3893228b29bc261b0c465eaf24b2353a0c78797abe087"}
Jul 08 19:45:01.965107 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.965017    2563 scope.go:117] "RemoveContainer" containerID="ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec"
Jul 08 19:45:01.975227 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.975204    2563 scope.go:117] "RemoveContainer" containerID="ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"
Jul 08 19:45:01.985504 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.985470    2563 scope.go:117] "RemoveContainer" containerID="ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec"
Jul 08 19:45:01.985908 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:45:01.985873    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec\": container with ID starting with ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec not found: ID does not exist" containerID="ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec"
Jul 08 19:45:01.986028 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.985919    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec"} err="failed to get container status \"ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec\": rpc error: code = NotFound desc = could not find container \"ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec\": container with ID starting with ca0e95f0af0044bec21c40ee71df901be50531510330e2bfa0275bdde796a2ec not found: ID does not exist"
Jul 08 19:45:01.986028 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.985949    2563 scope.go:117] "RemoveContainer" containerID="ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"
Jul 08 19:45:01.986318 ip-10-0-130-212 kubenswrapper[2563]: E0708 19:45:01.986292    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124\": container with ID starting with ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124 not found: ID does not exist" containerID="ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"
Jul 08 19:45:01.986377 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.986325    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124"} err="failed to get container status \"ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124\": rpc error: code = NotFound desc = could not find container \"ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124\": container with ID starting with ac70ef8aa802c36dd5eddf2135519cd564c57e86165c19c7e6f2f28ae1a99124 not found: ID does not exist"
Jul 08 19:45:01.988052 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.988032    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"]
Jul 08 19:45:01.993352 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.993326    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-235b05-openshift-default-7f8c4f58cf-vp2gj"]
Jul 08 19:45:01.999077 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:01.999057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-g6v25_d43b7087-c441-4777-bc7b-f003624d0fcf/copy-manifests/0.log"
Jul 08 19:45:02.005864 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:02.005841    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-g6v25_d43b7087-c441-4777-bc7b-f003624d0fcf/manager/0.log"
Jul 08 19:45:02.931661 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:02.931619    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" path="/var/lib/kubelet/pods/1aaa24f0-9d54-4009-9e42-ca9071bea0cb/volumes"
Jul 08 19:45:03.677304 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:03.677275    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ksgt9_4827bb5e-cd1e-45fd-a6b0-56044d60aa1a/manager/0.log"
Jul 08 19:45:03.919020 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:03.918990    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4vlhz_fb0aa254-de61-4722-bdcd-8429b56125a1/registry-server/0.log"
Jul 08 19:45:04.040890 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:04.040809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c6p9g_039f8156-8b28-45ef-8b3f-8ca26ec8417a/manager/0.log"
Jul 08 19:45:04.768674 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:04.768641    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r8wsd_9eba5102-c998-4dec-ae2b-237d3534f892/discovery/0.log"
Jul 08 19:45:04.886240 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:04.886207    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-2j55g_6644d20a-3b8d-4677-ab0e-30677ba8e54d/kube-auth-proxy/0.log"
Jul 08 19:45:05.116134 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:05.116076    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/1.log"
Jul 08 19:45:05.233132 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:05.233101    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mbpvb_e40b6fb8-5369-4520-bb1c-edfbb53d245f/payload-pre-processing/0.log"
Jul 08 19:45:05.354727 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:05.354686    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-sk8l9_15a0528a-79bf-4b59-b496-079fbe68da50/payload-processing/0.log"
Jul 08 19:45:06.876065 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:06.876031    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rnsfs_fbb9f849-3835-4632-9a70-a253b32e5b75/main/0.log"
Jul 08 19:45:06.989251 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:06.989217    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-7q8fd_8e8eb85b-2676-4a71-baa8-981698af7a3d/main/0.log"
Jul 08 19:45:07.221450 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:07.221370    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfkjv7_7acdca82-402e-4340-8c44-53aff3891538/main/0.log"
Jul 08 19:45:14.501365 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:14.501328    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4sk5h_bcc2336f-9b1b-401d-89a6-dd190e97c7b5/global-pull-secret-syncer/0.log"
Jul 08 19:45:14.674046 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:14.674006    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f2mkg_8b2f1675-9c4c-40dc-893e-6dd1ed09aecb/konnectivity-agent/0.log"
Jul 08 19:45:14.770762 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:14.770660    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-212.ec2.internal_991d6b5a2a481334cc1eaf58c5d1d435/haproxy/0.log"
Jul 08 19:45:20.004950 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:20.004918    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ksgt9_4827bb5e-cd1e-45fd-a6b0-56044d60aa1a/manager/0.log"
Jul 08 19:45:20.087856 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:20.087828    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4vlhz_fb0aa254-de61-4722-bdcd-8429b56125a1/registry-server/0.log"
Jul 08 19:45:20.191151 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:20.191100    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c6p9g_039f8156-8b28-45ef-8b3f-8ca26ec8417a/manager/0.log"
Jul 08 19:45:22.157738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.157655    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-w5kkn_c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c/kube-state-metrics/0.log"
Jul 08 19:45:22.174997 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.174972    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-w5kkn_c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c/kube-rbac-proxy-main/0.log"
Jul 08 19:45:22.194350 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.194314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-w5kkn_c8fcddc1-6fa4-4c8e-bd0f-ac0ba0a27a8c/kube-rbac-proxy-self/0.log"
Jul 08 19:45:22.248021 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.247984    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-skl47_d07ea37b-58da-4500-a70b-6b8eb46fa373/monitoring-plugin/0.log"
Jul 08 19:45:22.352737 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.352712    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjmck_0e1ff07f-b3bc-4464-b389-8c9117774155/node-exporter/0.log"
Jul 08 19:45:22.379737 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.379708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjmck_0e1ff07f-b3bc-4464-b389-8c9117774155/kube-rbac-proxy/0.log"
Jul 08 19:45:22.400924 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.400893    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjmck_0e1ff07f-b3bc-4464-b389-8c9117774155/init-textfile/0.log"
Jul 08 19:45:22.498295 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.498214    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8pvfz_70a7cdd0-4166-43c4-9fa7-bf7efde42935/kube-rbac-proxy-main/0.log"
Jul 08 19:45:22.520238 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.520208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8pvfz_70a7cdd0-4166-43c4-9fa7-bf7efde42935/kube-rbac-proxy-self/0.log"
Jul 08 19:45:22.541345 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.541319    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8pvfz_70a7cdd0-4166-43c4-9fa7-bf7efde42935/openshift-state-metrics/0.log"
Jul 08 19:45:22.578355 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.578328    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/prometheus/0.log"
Jul 08 19:45:22.599247 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.599221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/config-reloader/0.log"
Jul 08 19:45:22.625995 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.625964    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/thanos-sidecar/0.log"
Jul 08 19:45:22.646801 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.646779    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/kube-rbac-proxy-web/0.log"
Jul 08 19:45:22.669266 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.669246    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/kube-rbac-proxy/0.log"
Jul 08 19:45:22.690647 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.690623    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/kube-rbac-proxy-thanos/0.log"
Jul 08 19:45:22.711685 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:22.711658    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e9952d8-fe52-454e-ab54-96286003e799/init-config-reloader/0.log"
Jul 08 19:45:24.239636 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.239608    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-4flzn_63deeaca-7675-41e6-b345-191f40b70d31/networking-console-plugin/0.log"
Jul 08 19:45:24.538123 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538019    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"]
Jul 08 19:45:24.538549 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538518    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy"
Jul 08 19:45:24.538549 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538547    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy"
Jul 08 19:45:24.538742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538574    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy"
Jul 08 19:45:24.538742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538585    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy"
Jul 08 19:45:24.538742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538598    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy"
Jul 08 19:45:24.538742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538607    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff1b4df-ece7-4d37-b0a7-f24bfa806b4d" containerName="istio-proxy"
Jul 08 19:45:24.538742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538729    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy"
Jul 08 19:45:24.538742 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.538743    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aaa24f0-9d54-4009-9e42-ca9071bea0cb" containerName="istio-proxy"
Jul 08 19:45:24.542755 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.542734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.545738 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.545718    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjlc8\"/\"kube-root-ca.crt\""
Jul 08 19:45:24.545874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.545777    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjlc8\"/\"openshift-service-ca.crt\""
Jul 08 19:45:24.545874 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.545782    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sjlc8\"/\"default-dockercfg-mwzxk\""
Jul 08 19:45:24.551335 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.551311    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"]
Jul 08 19:45:24.639791 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.639755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-proc\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.639791 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.639796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-podres\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.640013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.639876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-sys\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.640013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.639897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5tj\" (UniqueName: \"kubernetes.io/projected/4afc40ad-d52b-4b0e-a49d-dd584f807207-kube-api-access-sl5tj\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.640013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.639978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-lib-modules\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742130 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-proc\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742334 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-podres\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742334 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-sys\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742334 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5tj\" (UniqueName: \"kubernetes.io/projected/4afc40ad-d52b-4b0e-a49d-dd584f807207-kube-api-access-sl5tj\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742496 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-lib-modules\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742615 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742593    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-lib-modules\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.742881 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.742862    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-proc\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.743111 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.743073    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-podres\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.743276 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.743262    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4afc40ad-d52b-4b0e-a49d-dd584f807207-sys\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.752264 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.752236    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5tj\" (UniqueName: \"kubernetes.io/projected/4afc40ad-d52b-4b0e-a49d-dd584f807207-kube-api-access-sl5tj\") pod \"perf-node-gather-daemonset-c9jnt\" (UID: \"4afc40ad-d52b-4b0e-a49d-dd584f807207\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.853488 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.853386    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:24.989019 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:24.988992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"]
Jul 08 19:45:24.991076 ip-10-0-130-212 kubenswrapper[2563]: W0708 19:45:24.991041    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4afc40ad_d52b_4b0e_a49d_dd584f807207.slice/crio-54f6290f63d2e304902d0fdd1aa9daa7604e18d35ed8b205695e678440aa3155 WatchSource:0}: Error finding container 54f6290f63d2e304902d0fdd1aa9daa7604e18d35ed8b205695e678440aa3155: Status 404 returned error can't find the container with id 54f6290f63d2e304902d0fdd1aa9daa7604e18d35ed8b205695e678440aa3155
Jul 08 19:45:25.089375 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:25.089305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt" event={"ID":"4afc40ad-d52b-4b0e-a49d-dd584f807207","Type":"ContainerStarted","Data":"54f6290f63d2e304902d0fdd1aa9daa7604e18d35ed8b205695e678440aa3155"}
Jul 08 19:45:25.384312 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:25.384271    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-fdff7d686-wfmnt_7ec56def-70d4-4264-a5c4-9bd8222a3014/console/0.log"
Jul 08 19:45:26.096043 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:26.096001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt" event={"ID":"4afc40ad-d52b-4b0e-a49d-dd584f807207","Type":"ContainerStarted","Data":"757b241df106d0bb0fcb6e85f5c4b6b309b2f9cb00e67a35c9b4aea1df9a961b"}
Jul 08 19:45:26.096242 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:26.096129    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:26.114664 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:26.114608    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt" podStartSLOduration=2.114592472 podStartE2EDuration="2.114592472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:26.113331054 +0000 UTC m=+2399.777893606" watchObservedRunningTime="2026-07-08 19:45:26.114592472 +0000 UTC m=+2399.779155022"
Jul 08 19:45:27.221871 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:27.221844    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zfw8l_b806e0a5-276a-4d25-93ec-e909b812ab48/dns/0.log"
Jul 08 19:45:27.240866 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:27.240836    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zfw8l_b806e0a5-276a-4d25-93ec-e909b812ab48/kube-rbac-proxy/0.log"
Jul 08 19:45:27.260858 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:27.260834    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g2rm5_11b1cfea-7567-45ce-a015-d1947a311e07/dns-node-resolver/0.log"
Jul 08 19:45:27.823066 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:27.823015    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8668666774-m8zdk_16289e5d-6029-465a-b20d-7138d7ab4206/registry/0.log"
Jul 08 19:45:27.863523 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:27.863489    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t4wlb_4fb99271-9db5-4d6a-b3a9-152683b48ca5/node-ca/0.log"
Jul 08 19:45:28.909297 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:28.909264    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r8wsd_9eba5102-c998-4dec-ae2b-237d3534f892/discovery/0.log"
Jul 08 19:45:28.931112 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:28.931068    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-2j55g_6644d20a-3b8d-4677-ab0e-30677ba8e54d/kube-auth-proxy/0.log"
Jul 08 19:45:28.982599 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:28.982560    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/0.log"
Jul 08 19:45:29.008422 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:29.008389    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/1.log"
Jul 08 19:45:29.045326 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:29.045294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mbpvb_e40b6fb8-5369-4520-bb1c-edfbb53d245f/payload-pre-processing/0.log"
Jul 08 19:45:29.091896 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:29.091868    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-sk8l9_15a0528a-79bf-4b59-b496-079fbe68da50/payload-processing/0.log"
Jul 08 19:45:29.817334 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:29.817309    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2k28s_35c3e76c-462d-4501-a8b7-e6483368606a/serve-healthcheck-canary/0.log"
Jul 08 19:45:30.388428 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:30.388397    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qw687_96d21489-9846-4fac-aa04-91845fd9e46b/kube-rbac-proxy/0.log"
Jul 08 19:45:30.409558 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:30.409532    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qw687_96d21489-9846-4fac-aa04-91845fd9e46b/exporter/0.log"
Jul 08 19:45:30.430775 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:30.430749    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qw687_96d21489-9846-4fac-aa04-91845fd9e46b/extractor/0.log"
Jul 08 19:45:32.112097 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:32.112056    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-c9jnt"
Jul 08 19:45:32.602800 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:32.602713    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2frh5_fea46e71-eb0d-4ad6-96a9-94be06d0d4dd/manager/0.log"
Jul 08 19:45:32.869590 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:32.869561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-598545dc6b-ncr96_9f4ae60b-2c83-40c5-a2c9-4fb477d05a00/manager/0.log"
Jul 08 19:45:33.063035 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:33.063005    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-g6v25_d43b7087-c441-4777-bc7b-f003624d0fcf/manager/0.log"
Jul 08 19:45:33.084859 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:33.084836    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-g6v25_d43b7087-c441-4777-bc7b-f003624d0fcf/copy-manifests/0.log"
Jul 08 19:45:35.169596 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:35.169561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r8wsd_9eba5102-c998-4dec-ae2b-237d3534f892/discovery/0.log"
Jul 08 19:45:35.182493 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:35.182463    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-2j55g_6644d20a-3b8d-4677-ab0e-30677ba8e54d/kube-auth-proxy/0.log"
Jul 08 19:45:35.226654 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:35.226625    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/0.log"
Jul 08 19:45:35.253355 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:35.253315    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/1.log"
Jul 08 19:45:35.283569 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:35.283526    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mbpvb_e40b6fb8-5369-4520-bb1c-edfbb53d245f/payload-pre-processing/0.log"
Jul 08 19:45:35.319525 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:35.319436    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-sk8l9_15a0528a-79bf-4b59-b496-079fbe68da50/payload-processing/0.log"
Jul 08 19:45:36.587346 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:36.587311    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r8wsd_9eba5102-c998-4dec-ae2b-237d3534f892/discovery/0.log"
Jul 08 19:45:36.600588 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:36.600563    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-2j55g_6644d20a-3b8d-4677-ab0e-30677ba8e54d/kube-auth-proxy/0.log"
Jul 08 19:45:36.642452 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:36.642067    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/0.log"
Jul 08 19:45:36.669302 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:36.669273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9mvc4_566a6cc7-9156-43fe-a1fa-853634067c32/istio-proxy/1.log"
Jul 08 19:45:36.700013 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:36.699980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mbpvb_e40b6fb8-5369-4520-bb1c-edfbb53d245f/payload-pre-processing/0.log"
Jul 08 19:45:36.735201 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:36.735169    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-sk8l9_15a0528a-79bf-4b59-b496-079fbe68da50/payload-processing/0.log"
Jul 08 19:45:41.371797 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.371767    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/kube-multus-additional-cni-plugins/0.log"
Jul 08 19:45:41.391980 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.391954    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/egress-router-binary-copy/0.log"
Jul 08 19:45:41.411991 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.411963    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/cni-plugins/0.log"
Jul 08 19:45:41.432032 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.432005    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/bond-cni-plugin/0.log"
Jul 08 19:45:41.451686 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.451657    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/routeoverride-cni/0.log"
Jul 08 19:45:41.470688 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.470664    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/whereabouts-cni-bincopy/0.log"
Jul 08 19:45:41.490473 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.490440    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmmcp_1677a6e8-b5a9-42a2-8a07-ce4b33ed3328/whereabouts-cni/0.log"
Jul 08 19:45:41.680091 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.680047    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wtcj7_41698256-87d5-433e-9e4f-a3f7de1fbfc3/kube-multus/0.log"
Jul 08 19:45:41.744760 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.744733    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kxb2x_69ca6196-d827-45d8-985d-b0d8ffcd0f57/network-metrics-daemon/0.log"
Jul 08 19:45:41.768004 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:41.767977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kxb2x_69ca6196-d827-45d8-985d-b0d8ffcd0f57/kube-rbac-proxy/0.log"
Jul 08 19:45:43.032361 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.032323    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/ovn-controller/0.log"
Jul 08 19:45:43.048027 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.047999    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/ovn-acl-logging/0.log"
Jul 08 19:45:43.074959 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.074924    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/kube-rbac-proxy-node/0.log"
Jul 08 19:45:43.097231 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.097210    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/kube-rbac-proxy-ovn-metrics/0.log"
Jul 08 19:45:43.116284 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.116258    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/northd/0.log"
Jul 08 19:45:43.135882 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.135852    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/nbdb/0.log"
Jul 08 19:45:43.156367 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.156340    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/sbdb/0.log"
Jul 08 19:45:43.208167 ip-10-0-130-212 kubenswrapper[2563]: I0708 19:45:43.208130    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wvxjv_a9c3b6fe-289b-427e-bb03-831dae993b10/ovnkube-controller/0.log"